Open in new window / Try shogun cloud
--- Log opened Fri Jun 10 00:00:03 2016
--- Day changed Fri Jun 10 2016
@lambdayOXPHOS: well, ENABLE_GPULIB thing has to go..00:00
@lambdayOXPHOS: now we have register_backend thing00:00
OXPHOSbut users will have another input ENABLE_GPU_CALCULATION, to ask Shogun to choose from any GPULIB if there is any00:00
@lambdayif there are two gpu backends, users can only register one00:00
@lambdaywhichever is registered, that is enabled - that should be the principle00:01
OXPHOSwill shogun throw errors if user try to register two?00:01
OXPHOSthese will all be in cmake right00:02
@lambdayOXPHOS: well, we should decide that a bit carefully.. say, for a part of his application, he wants to use viennacl.. so he registers that.. then in some later part of the code he wants to use CUDA.. so he registers that00:03
@lambdayumm.. not sure about that00:03
@lambdayOXPHOS: shouldn't it be from code itself?00:03
@lambdaycmake would just find supported libs00:03
@lambdaywell, no.. maybe you have a point00:04
OXPHOSlambday: well if it can be changed..guess a global variable would work?00:04
@lambdayOXPHOS: global variable for what?00:04
@lambdayOXPHOS: we already have that inside sg_linalg.m_gpubackend, no?00:05
@lambdaythat's pretty global :D00:05
OXPHOSlambday: yeah you'r right. It is actually global because of sg_linalg. So since there's only one SGLinalg, there IS only one GPUBackend at one time00:06
@lambdayOXPHOS: in my mind, registering backend was about setting this m_gpubackend..00:06
@lambdayOXPHOS: but not sure whether a cmake option could be set..00:06
@lambdaysomething like, if cmake finds any supported gpu backend, that thing gets registered automatically..00:07
@lambdayso if there are multiple, there has to be a priority - which one to use as default00:07
@lambdaybut the user should always be able to change this using sg_linalg.register_gpu_backend00:07
OXPHOSthere's a problem with current m_gpubackend00:09
@lambdayOXPHOS: which is?00:09
OXPHOSit can be named whatever00:09
OXPHOSlike whatever library you try to register , it'll stay the same00:10
lisitsynok let me try to fix it00:10
@lambdaybut you changed it to shared ptr, no?00:10
OXPHOSi mean00:10
@lambdaylisitsyn: haha sleeping tiger woke up :D00:11
OXPHOSnow is now #ifdef HAVE_VIENNACL, do blabla; #elif HAVE_CUDA, do blabla00:11
@lambdayhah my Indian language background always makes me read things exactly as they are spelled :D00:11
@lambdayif someone says ja, I actually ja instead of ha xD00:12
@lambdayI actually read**00:12
@lambdayjabit, jou see00:12
@lambdaysurprisingly, j = h, and j = y00:12
@lambdaysanskrit doesn't have that :D00:13
@lambdayhah my Indian language background always makes me read things exactly as they are spelled :D00:13
OXPHOSjeah i know there're several languages(?) in India00:14
OXPHOSnever know the difference :)00:14
@lambdayOXPHOS: neither do we :D00:14
@lambdayI myself know like 3 of them00:14
@lambdayhah my Indian language background always makes me read things exactly as they are spelled :D00:14
@lambdaywell, 2.67 to be exact00:15
OXPHOSI have a friend from India who only speaks English00:15
@lambdayI know, right? B-)00:15
@lambdayOXPHOS: he's lying00:15
@lambdayshe's lying? :/00:16
@lambdayah english needs a gender neutral pronoun00:16
OXPHOSreally - she said English is her first language - might not be the only one00:17
OXPHOSwe distinguish that in Chinese also. only in written. weird00:17
@lambdayOXPHOS: I am yet to see an Indian (who stayed in India) whose first language is English..00:18
OXPHOSI'll question her next time;)00:19
@lambdayOXPHOS: just ask which part of India she's from and I'll tell you her native language ;)00:19
@lambday(almost) every state has a different one00:20
OXPHOSso you all speak english00:20
OXPHOSborn bilingual00:21
OXPHOSbut anyway - it doesn't matter how many gpubackend you try to register now, they will be the SAME one00:22
OXPHOSbecause the switch between backends is dependent on #ifdef HAVE_GPULIB, which is set at cmake00:23
@lambdayOXPHOS: no! there can be multiple of those.. but at a given point there can be only one00:23
@lambdaysee what I mean?00:23
@lambdayso thing to keep in mind - make register_gpu_backend() and register_cpu_backend() thread safe00:24
@lambdayOXPHOS: ^00:24
@lambdayvery important!00:24
OXPHOSi see00:24
OXPHOSBUT say you're now (supposed to be) using ViennaCL gpubackend and you want to switch to CUDA, how can you do that?00:26
@lambdayalso, I am trilingual actually.. hindi, bengali and english.. a bit punjabi.. can understand gujarati, assamese, oriya, marathi.. xD00:26
@lambdayOXPHOS: sg_linalg.register_gpu_backend(new CUDABackend()) or something00:27
lisitsynI am half lingual00:27
@lambdayah I see what you mean00:27
@lambdayOXPHOS: so the pimpl allows you to have only one impl at a time due to those ifdefs00:28
OXPHOSyou speak C++, python, bash ...00:28
@lambdayOXPHOS: when lisitsyn was born, his first words here `std::cout << "hello world!" << std::endl;'00:28
lisitsyndo you know what's the best interview question on sed?00:28
lisitsyndo you know some sed?00:29
@lambdaylisitsyn: you're asking or you're about to tell yourself?00:29
lisitsynif he/she knows reject00:29
lisitsynbecause he is going to write everything in sed00:29
lisitsynobject relational mapping? sounds like a good sed task00:29
@lambdaylisitsyn: haha xD00:30
OXPHOSi can guess the point..00:31
OXPHOSlambday: yeah i think so.. I mean it is "static" actually. there's no CUDABackend(). there's only one GPUBackend() which can work with whatever GPUlibrary Shogun can use.00:31
OXPHOSnot whatever the users wants to work with00:31
@lambdayOXPHOS: yeah I see.. so it doesn't support multiple backends yet..00:32
@lambdaymaybe we should support them later00:32
OXPHOSyes I think this is another branch00:33
@lambdayOXPHOS: not too hard to change that.. basically what we have to do is to make GPUBackend a base class.. and provide different impls like ViennaCLBackend, CUDABackend etc..00:33
@lambdaybut that's not the focus for now00:33
OXPHOSlambday: we were talking about the GPU_CALCULATION stuff.. so I'll leave it there for now?00:34
@lambdaymaybe we'll never support naked cuda..00:34
@lambdayOXPHOS: I didn't fully get what you wanted to do with this macro :(00:35
@lambdaywhy would this be needed again?00:35
OXPHOSlet's say shogun support GPULIB1 and 2. The user has GPULIB1 and 2. The user registered GPULIB1. but for some reason, he doesn't want to use GPUBackend for calculation00:36
@lambdayOXPHOS: this problem will be solved **if** we allow them to register different backends..00:37
@lambdayas long as that is not supported, this is not possible00:37
OXPHOSokay so i can just use HAVE_VIENNACL for now00:37
@lambdayOXPHOS: yeah that should be fine..00:38
OXPHOSnot fancy :)00:38
@lambdaywell it's easy to change that you see.00:38
@lambday(a) make GPUBackend an abstract base class (b) provide a subclass ViennaCLBackend as a concrete impl.. do the ifdefs there00:39
@lambdaynot in the header though00:40
@lambdayso ViennaCLBackend would be what you have as GPUBackend for now00:40
@lambdaypimpl.. and throws error if no viennacl is found00:41
@lambdayactually you know what, I like this idea!00:41
@lambdayso sg_linalg.m_gpubackend would be of type shared_ptr<GPUBackend>, and when the user registers, he does sg_linalg.register_gpu_backend(new ViennaCLBackend())00:42
OXPHOSvery "engineering"00:42
@lambdaynow if that bastard doesn't have viennacl in the first place, we fail him in the constructor itself00:43
@lambdaybut it all has to be pimpl00:43
@lambdaylisitsyn: ^ could you please comment?00:44
@lambdayOXPHOS: his brain is doing something `unique' at this moment I can imagine00:45
OXPHOSlambday: raw pointer00:45
OXPHOSsee whether this works ;)00:45
@lambdaytag him00:45
@lambdaythen it will xD00:45
OXPHOSso i can do now a. refactor gpubackend class; b. add other SGVector methods; c. add BaseMatrix/CPUMatrix/GPUMatrix..00:46
OXPHOSany priority?00:46
@lambdayOXPHOS: what do you mean by other SGVector methods00:47
OXPHOSlike sum/mean00:47
@lambdayas in mean(), l2norm() etc?00:47
@lambdayI see00:47
@lambdayyeah so I'll say, b, c, a00:48
@lambdaybut if lisitsyn replies quickly, then maybe a next if you want00:48
@lambdaylisitsyn: CAnIdea* idea = new CAnIdea();00:49
OXPHOSyes the tons of methods in linalg should all be migrated someday00:50
@lambdayOXPHOS: we only have a few there ;)00:52
@lambdayOXPHOS: we gotta add many more00:52
OXPHOSlambday: the kilograms of methods ;)00:53
@lambdayOXPHOS: hah one problem with making GPUBackend an abstract class is that we gotta make the methods virtual.. which is not possible since those are templates01:01
@lambdayOXPHOS: maybe it is not as straightforward as I thought..01:01
@lambdayso let's not worry about that now01:02
@lambdaymaybe we can use Any :/01:03
@lambdayyeah Any would work.. the calls are going through SGLinalg anyway.. so in that we just to recall_type01:04
-!- lambday [6d9941c0@gateway/web/freenode/ip.] has quit [Ping timeout: 250 seconds]01:11
-!- lambday [6d9941c0@gateway/web/freenode/ip.] has joined #shogun01:13
-!- mode/#shogun [+o lambday] by ChanServ01:13
-!- lambday [6d9941c0@gateway/web/freenode/ip.] has quit [Ping timeout: 250 seconds]01:17
-!- OXPHOS [9d8b1501@gateway/web/freenode/ip.] has quit [Ping timeout: 250 seconds]01:36
-!- mizari [] has joined #shogun03:22
-!- sonne|work [~sonnenbu@] has quit [Quit: Leaving.]08:40
-!- sonne|work [~sonnenbu@] has joined #shogun08:41
-!- sanuj [~sanuj@] has joined #shogun08:41
-!- sanuj [~sanuj@] has quit [Ping timeout: 260 seconds]09:03
-!- sanuj [~sanuj@] has joined #shogun09:33
-!- HeikoS [] has joined #shogun09:39
-!- mode/#shogun [+o HeikoS] by ChanServ09:39
-!- HeikoS [] has quit [Client Quit]09:40
@wikingSaurabh7: here?10:36
Saurabh7wiking: yes10:40
@wikingSaurabh7: ok so10:42
@wikingwould it be hard to extends for example10:42
@wikingthe LARS (or any other benchmarks) with mem usage?10:42
Saurabh7wiking: not sure, how would you check for this one ?10:43
Saurabh7if you point me somewher i can do10:43
Saurabh7how done till now10:43
Saurabh7btw for lars I used a bufer matrix to acces active columns runtiem went form 18 -> 1210:44
Saurabh7I am extending it for inactive colmns too, it should go lower10:45
@wikingok i'll try to put together a small howto10:45
@wikingfor measuring mem usage of things10:45
@wikingbecause actually that'd be interesting as well10:45
Saurabh7yes definetly10:45
Saurabh7wiking: I used this for kmeans once
Saurabh7I can try this dunno if its reliable10:50
-!- HeikoS_mobile [] has joined #shogun10:58
HeikoS_mobilewiking: jo10:59
Saurabh7HeikoS_mobile: hi11:04
HeikoS_mobileSaurabh7: Ho11:05
HeikoS_mobileI'm on phone in train11:05
Saurabh7HeikoS_mobile: I used yesterdays idea to remove loops, got good gains11:05
HeikoS_mobileSo limited typing11:05
HeikoS_mobileGot some numbers?11:05
Saurabh7let me put in the thread ?11:06
Saurabh7Now I ahve to use it for inactive columns aswell11:06
Saurabh7that will remove teh last loop11:06
HeikoS_mobileMemcpy is fast :)11:07
Saurabh7HeikoS_mobile: Didnt ahve to use memcpy11:07
sanujHeikoS_mobile, hey11:08
sanujHeikoS_mobile, when will you have your laptop? we need to merge some PRs today.....11:09
-!- HeikoS_mobile [] has quit [Quit: Mutter:]11:10
-!- HeikoS_mobile [] has joined #shogun11:17
-!- besser82 [~besser82@fedora/besser82] has joined #shogun11:20
-!- mode/#shogun [+o besser82] by ChanServ11:20
sanujHeikoS_mobile, hey11:21
-!- HeikoS_mobile [] has quit [Remote host closed the connection]11:23
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 260 seconds]11:27
-!- HeikoS_mobile [] has joined #shogun11:31
HeikoS_mobilesanuj: will merge things later today11:31
sanujHeikoS_mobile, okay!11:32
HeikoS_mobileSaurabh7: can you send me an email with the numbers? Sorry I have a laggy connection11:32
HeikoS_mobileThere is still a proven with r and static calls unfortunately, I might just disable it for now if I can't fix11:32
-!- HeikoS_mobile [] has quit [Remote host closed the connection]11:35
-!- sanuj [~sanuj@] has quit [Ping timeout: 264 seconds]12:02
-!- sanuj [~sanuj@] has joined #shogun12:15
sanujhey lisitsyn12:56
sanujgot time?12:56
arianepaolahello everyone13:46
arianepaolahi wiking13:57
arianepaolawiking: did you check the gist, I sent yesterday13:58
@wikingmmm sorry remember i'm utc+813:59
@wikingso i think i've missed it13:59
arianepaolano problem14:00
arianepaolaI will check in the logs14:00
@wikingcan you paste again?14:00
arianepaolawiking: when you have time, check this gist
arianepaolawiking: the in the PR overloads functionality from distutils to bootstrap the cmake configuration process and the compilation18:1714:01
arianepaolawiking: the gist just deals with the Python related content in lib. you can test it building shogun manually and then in the build dir: python bdist14:01
arianepaolaarianepaola^^ HeikoS: if you want to check also14:01
@wikingthis looks awseom14:01
@wikingwhat happens on sdist? :)14:01
arianepaolait packs the files14:02
@wikingbut anyhow this is great stuff!14:02
arianepaolabut you can specify with MANIFEST.in14:02
@wikingbecause actually we can add to the repo a
@wikingthat cmake can generate the right one14:02
arianepaolain the logs, starting 14:41
@wiking(see version)14:02
@wikingi'm just wondering what would be the best way to distribute the setup.py14:03
arianepaolathe PR with the cmake bootstrap overwrites distutils classes and this gist uses minimal of setuptools to make it work14:03
arianepaolaso I have to test line by line and see where each one breaks14:03
@wikingand not hardcode the version and stuff in it14:03
arianepaolajust put in the repo14:03
@wikingyep but then what do we do with VERSION14:03
@wikingnow it's hardcoded14:04
@wikingideally it shouldn't be :)14:04
@wikingmmm we could use the regexp14:04
@wikingthat cmake uses14:04
@wikingto get the version, right?14:04
arianepaolafor the version I had this question yesterday:14:04
arianepaolaarianepaolalisitsyn: hi, I have a question regarding src/shogun/lib/versionstring.h14:04
arianepaolaarianepaolalisitsyn: is there a way to check if we are using git source? e.g. will the version change for a Release version?14:04
arianepaolaarianepaolaI want only to append the info from #define VERSION_REVISION 0x83970a7 when it is a development version14:04
arianepaolaif setuptools does not complain, reading the file and a regular expression matching the version should work14:05
arianepaolait depends on when setuptools gets the version, e.g. variable content passed14:06
arianepaolaI think at the end when bdist, but then we have to check for sdist14:06
@wikingso as you can see14:07
@wikingif there's no .git14:07
@wikingthen we use NEWS info14:07
@wikingto fill out those macros14:07
arianepaolathen the result goes in src/shogun/lib/versionstring.h, right?14:08
arianepaolaregarding cmake generating the setup.py14:09 bootstraps cmake14:10
arianepaolaso if it should generate from, it would be: cmake -> -> cmake -> make -> package14:10
arianepaolaif it works to read the versionstring, without setuptools complaining14:10 -> cmake -> make -> package14:10
arianepaolawiking: other idea for version passing to setuptools is to pass the return of a method to version14:13
arianepaolawiking: this method then can check if cmake was run already or not, then run build, for the build step it will just check if cmake was configure to avoid double work14:14
@wikingyeah this is a bit problematic :S14:15
@wikingor just we keep rolling the static version line there in setup.py14:15
@wikingwhen we do a release14:15
@wikingbit ugly because you need to do then the double the work14:16
@wikingbut i think reading file + regex for a version14:16
@wikingshould be ok from setuptools or?14:16
arianepaolabut it depends when setuptools fills in the version14:17
arianepaolaso probably it will be versio = get_shogun_version()14:17
arianepaolaif ran_cmake: return parse_version_string()14:18
arianepaola else: run_cmake()  + return parse_version_string()14:19
arianepaola^^ wiking: something like this should work14:19
arianepaolawiking: but hey, only setuptools knows :-)14:20
arianepaolaI am going to work on this part now14:20
@wikinggood work keep up with it!14:21
arianepaolaI will update the gist14:21
sanujlisitsyn, there?15:17
lisitsynsanuj: yeap15:17
sanujlisitsyn, can you have a look at this
sanujthis is not working for python15:18
lisitsynI can't guess anything from what you said :D15:19
sanujlisitsyn, any tag and get/set are not visible in python15:20
sanujany, tag*15:20
sanujit build succesfully (without errors) but above things are not visible in python15:23
sanujlike if i do15:23
lisitsynok ok interesting15:23
sanujlisitsyn, what do you think am I missing?15:25
lisitsynsanuj: comparing to other template swig things15:27
lisitsynit looks correct15:28
sanujlisitsyn, how to debug this?15:28
sanujlisitsyn, this was working with my prototype15:28
lisitsynsanuj: are you sure there is nothing in logs?15:35
lisitsynwarnings anything?15:35
sanujlisitsyn, oh i think i exported the wrong path15:36
sanujlet me check15:36
lisitsynand please show me how you call that15:36
sanujlisitsyn, sorry :D15:38
sanujit was the wrong path15:38
sanujlisitsyn, call what?15:38
arianepaolawiking: what do you think?
sanujlisitsyn, got  time to chat about some details?15:41
arianepaolawiking: it gets the version now, but strangely it stopped packaging the rest of the modules15:42
-!- OXPHOS` [9d8b131c@gateway/web/freenode/ip.] has joined #shogun16:24
OXPHOS`wiking: there?16:42
-!- HeikoS_mobile [] has joined #shogun16:49
HeikoS_mobilesanuj: Saurabh7: back soon16:50
HeikoS_mobileSaurabh7: where are the new times? Couldn't find16:50
sanujHeikoS_mobile, will be back in 20 mins16:51
-!- sanuj [~sanuj@] has quit [Quit: Leaving]16:53
@wikingsoert of16:55
-!- HeikoS_mobile_ [] has joined #shogun16:58
-!- HeikoS_mobile [] has quit [Ping timeout: 276 seconds]16:58
-!- HeikoS_mobile_ [] has quit [Remote host closed the connection]17:01
-!- sanuj [~sanuj@] has joined #shogun17:24
arianepaolawiking: what do you think?
-!- lambday [6d9941c0@gateway/web/freenode/ip.] has joined #shogun17:30
-!- mode/#shogun [+o lambday] by ChanServ17:30
@wikinglooks good17:38
@wikingOXPHOS`: ping17:40
@wikingarianepaola: but we should use maybe rather NEWS to do the regex17:40
@wikingas the version string into the versionstring.h is being generated from there17:41
OXPHOS`wiking: hey17:41
OXPHOS`wiking: for the comments in linalg17:41
OXPHOS`do you mean we don't have to distinguish CPU/GPU Vector?17:42
arianepaolawiking: versionstring.h is probably less error prone, as cmake runs anyway before17:42
@wikingarianepaola: yeah but in this case you wouldnt need to have cmake run prior to this17:51
@wikingOXPHOS`: i was just wondering what is the reason for distinguishing17:52
OXPHOS`wiking: Yeah actually I can't think of any now..17:54
arianepaolawiking: I thought that also, but it is needed for the module content17:54
arianepaolapackage_dir and adata_files17:55
arianepaolacan only be generated after running cmake17:55
arianepaolawiking: leading to a chicken or the egg problem17:56
@wikingOXPHOS`: ok then17:59
-!- HeikoS [] has joined #shogun18:01
-!- mode/#shogun [+o HeikoS] by ChanServ18:01
@HeikoSSaurabh7: hi!18:01
@HeikoSsanuj: hi!18:01
@HeikoSarianepaola: hi!18:01
@HeikoSOXPHOS`: hi!18:01
sanujHeikoS, yooooo!!18:01
@HeikoSnow back on computer18:01
@HeikoSwith some time18:02
arianepaolahi HeikoS18:02
@HeikoSarianepaola: hi! how are things going on your side?18:02
OXPHOS`hello HeikoS18:03
@HeikoSOXPHOS`: hi how are things going?18:03
arianepaolaHeikoS: the mini version can detect now the shogun version. somehow it is now missing the module files, that I am looking into. the previous gist included them.18:04
arianepaolaHeikoS: then I can merge it with the cmake bootstrapping code18:04
@HeikoSarianepaola: cool about the version, checking the gist atm18:04
@HeikoSyeah cool18:04
@HeikoSwhat do you mean by module files?18:04
arianepaolaHeikoS: see at the end the output of this one:
-!- shogun-notifier- [] has joined #shogun18:05
shogun-notifier-shogun: Sanuj :develop * de25478 / / (10 files):
shogun-notifier-shogun: add setters/getters in nn and autoencoders18:05
shogun-notifier-shogun: Heiko Strathmann :develop * d7ac736 / / (10 files):
shogun-notifier-shogun: Merge pull request #3276 from sanuj/nn_cleanup18:05
shogun-notifier-shogun: add setters/getters in nn and autoencoders18:05
sanujwohoo merged18:05
OXPHOS`HeikoS I updated the logistic regression:
OXPHOS`and the kmeans
arianepaolaHeikoS: e.g. dist-packages/ and then shogun/Classifier ... IO ... Loss .. Structure18:06
@HeikoSarianepaola: I see18:06
@HeikoSarianepaola: so these guys are not yet packaged into the archive18:07
shogun-buildbotbuild #709 of trusty - libshogun - viennacl is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>, Sanuj <>18:07
arianepaolaHeikoS: they are (see the other gist).18:07
@HeikoSsanuj: whats up with this gmm failure?18:07
shogun-buildbotbuild #3782 of deb1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>, Sanuj <>18:07
arianepaolaHeikoS: the strange thing is that it seems to change based on the cwd18:07
sanujHeikoS, we need to merge this to fix the gmm failure
arianepaolaeven though I get the same files18:08
@HeikoSarianepaola: strange indeed18:08
@HeikoSarianepaola: I guess google can help?18:08
arianepaola>>> glob.glob('install/lib/*/dist-packages')[0]18:08
arianepaola>>> glob.glob('install/lib/libshogun*')18:08
arianepaola['install/lib/', 'install/lib/', 'install/lib/']18:08
arianepaola>>> glob.glob('build/install/lib/*/dist-packages')[0]18:08
arianepaola>>> glob.glob('build/install/lib/libshogun*')18:08
arianepaola['build/install/lib/', 'build/install/lib/', 'build/install/lib/']18:08
sanujHeikoS, but R fails18:09
@HeikoSarianepaola: got any ideas what to do about this=?18:09
arianepaolaHeikoS: google not really, but it has to do how setuptools does get cwd18:09
arianepaolaI am trying to reproduce and overwrite it18:10
shogun-notifier-shogun: Sanuj :develop * 2549c77 / examples/meta/src/clustering/
shogun-notifier-shogun: init math for gmm cookbook integration data18:10
shogun-notifier-shogun: Heiko Strathmann :develop * 78b25b8 / examples/meta/src/clustering/
shogun-notifier-shogun: Merge pull request #3239 from sanuj/cookbook_gmm18:10
shogun-notifier-shogun: init math for gmm cookbook integration data18:11
@HeikoSarianepaola: ok cool18:11
-!- HeikoS [] has left #shogun []18:11
-!- HeikoS [] has joined #shogun18:11
-!- mode/#shogun [+o HeikoS] by ChanServ18:11
arianepaolaI am checking the source a t
@HeikoSarianepaola: good idea18:11
sanujHeikoS, R was failing for ^^^18:11
@HeikoSsanuj: I know18:11
@HeikoSI will deactivate it now18:11
@HeikoSand see how to fix it18:11
@HeikoSsome problem that I dont entirely understand18:12
arianepaolaHeikoS: seems there is find_packages, I will give it a try and see how that goes18:12
@HeikoSbut we need these static methods in there, otherwise it is a blocker18:12
shogun-buildbotbuild #710 of trusty - libshogun - viennacl is complete: Success [build successful]  Build details are at
@HeikoSha! ^18:12
shogun-buildbotbuild #3783 of deb1 - libshogun is complete: Success [build successful]  Build details are at
shogun-notifier-shogun: Heiko Strathmann :develop * ee8f2a5 / examples/meta/CMakeLists.txt:
shogun-notifier-shogun: temporarily disable R meta examples18:14
sanujHeikoS, can you review this
sanujit is merge ready18:14
@HeikoSsanuj: OXPHOS` I pushed a commit that disables R meta examples18:15
sanujreviewed by lisitsyn and lambday18:15
@HeikoSsanuj: will check soon18:15
sanujif you have some changes, i can do and squash the commits18:15
sanujHeikoS, cool!18:15
@lambdayHeikoS: hey18:16
@HeikoSlambday: yo!18:16
@lambdaysanuj: hello18:16
@HeikoSIm in the office now btw18:16
@HeikoSlambday: I guess youre home?18:16
@lambdayHeikoS: yeah I stayed at home18:16
sanujand HeikoS this is the updated mkl cookbook
sanujfails for R and Ruby18:16
@HeikoSlambday: cool, how is xvalidation going?18:16
arianepaolaHeikoS: does Shogun build Python 3 modules?  As I am working around the limitation of lacing PEP420 support in setuptools (they added this for Python >3.3
sanujplease look at the errors once, if you want me to check on local then I'll install R and ruby on my laptop18:17
@HeikoSsanuj: give me a sec, too many things at once atm :) ping me on this in 30 mins18:17
sanujlambday, yooo18:17
sanujHeikoS, sure :)18:17
@HeikoSarianepaola: I think it does18:17
sanujbtw you work in office? and not in a lab?18:17
OXPHOS`HeikoS: cool! then the kmeans is ready to merge then18:17
@HeikoSarianepaola: see
@HeikoSsanuj: same thing :)18:18
arianepaolaI will make it work on Python2 first and then check what fails for 318:18
@HeikoSarianepaola: you can check the config and output logs there18:18
@HeikoSarianepaola: yeah good approach18:18
@HeikoSunfortunately, python2->python3 is not trivial18:18
@HeikoSdont know who the hell decided that18:18
@HeikoSbut its such a shot in the knee18:18
@HeikoSfor Python18:18
@HeikoSreally a shame18:18
@HeikoSbut well18:18
@HeikoS(and they didnt even speed it up)18:19
arianepaolaAt least it is not so bad as Swift 2.0 -> 2.1 -> 2.3 -> 318:19
arianepaola* 2.1 -> 2.218:19
shogun-buildbotbuild #47 of xenial - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Sanuj <>18:19
@HeikoSdont know about that, what happened there?18:19
arianepaolaHeikoS: they change their mind every time ->
@lambdayHeikoS: not much yet. wondering whether I should do the translational invariant thing first or just test the way I did for multi kernel MMD for max MMD kernel selection18:20
@HeikoSarianepaola: omg scary18:20
@HeikoSlambday: I can you explain more?18:20
-!- HeikoS [] has left #shogun []18:21
-!- HeikoS [] has joined #shogun18:21
-!- mode/#shogun [+o HeikoS] by ChanServ18:21
@lambdayHeikoS: yo..18:21
@HeikoSlambday:  yo again :)18:21
shogun-buildbotbuild #2907 of bsd1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Heiko Strathmann <>, Sanuj <>18:21
sanujwhy do these things fail18:21
@lambdayHeikoS: so for the max MMD thing, I kinda did a hack.. going through the distance matrix once and computing the kernel function(s) on it18:22
sanujlast to last one was for lda unit tests18:22
@lambdaywithout having adding the classes yet that we discussed about18:22
@HeikoSsanuj: check the build ihstory18:22
@HeikoSthere is another problem18:22
@HeikoSnot because of the recent merge18:23
@HeikoSlambday: I guess the best way would be to prototype first using ad-hoc distance precomputation18:23
@HeikoSlambday: and then build your internals around that18:23
shogun-buildbotbuild #48 of xenial - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>18:23
@HeikoSlambday: then once it works, put the translation invariant stuff into shogun18:23
@lambdayHeikoS: alright18:24
sanujbtw i just remembered something18:30
@lambdayOXPHOS`: yo18:31
@lambdayso we can have everything in a single Vector class, right?18:31
sanujlambday, did you tell heiko how my name is pronounced :P18:31
@lambdaywhich we'll pass around as const ref instead of a shared ptr18:31
@lambdayOXPHOS`: and make it use a forward declared gpuarray..18:32
@lambdayso none of these things will be in the header of Vector, but in the cpp.. and then in GPUBackend, you just use that..18:32
@lambdaysanuj: haha yeah18:32
OXPHOS`lambday: I guess so..what's that about gpuarray? I feel gpuarray doesn't need any change18:33
@lambdayOXPHOS`: things to think about : when you want to get a gpu vector from the factory, you set the ongpu flag true18:34
@lambdayOXPHOS`: nah..18:34
@lambdayOXPHOS`: gpuarray is fine18:34
OXPHOS`lambday: sure18:35
arianepaolawiking and HeikoS: seems it is the where='.' in
arianepaolawhere is undocumented18:37
-!- besser82 [~besser82@fedora/besser82] has joined #shogun18:38
-!- mode/#shogun [+o besser82] by ChanServ18:38
@HeikoSarianepaola: well spotted!18:38
@HeikoSarianepaola: cool ,so will that fix it?18:42
arianepaolait includes the submodules and, but it is not including that was included before18:44
arianepaolaas package data18:44
arianepaolawiking and HeikoS: see
arianepaolanow it is about getting back in18:48
-!- OXPHOS` [9d8b131c@gateway/web/freenode/ip.] has quit [Ping timeout: 250 seconds]18:48
-!- travis-ci [] has joined #shogun18:52
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []18:52
@HeikoSsanuj: hi18:58
@HeikoSwhat was the discussion about the spaces?18:58
@HeikoSI am just looking at the tag PR18:58
sanujHeikoS, lisitsyn told me to keep spaces18:58
sanujx = y and not x=y18:58
@HeikoSlisitsyn: can you remind me why?18:58
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 272 seconds]18:58
@HeikoSand also, we should update our style guidelines then18:59
@HeikoSnot that it really matters :)18:59
sanujHeikoS, he likes it18:59
-!- besser82 [~besser82@fedora/besser82] has joined #shogun19:00
-!- mode/#shogun [+o besser82] by ChanServ19:00
shogun-buildbotbuild #36 of FC23 - libshogun - aarch64 is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>, Sanuj <>19:07
@HeikoSsanuj: question:19:11
@HeikoSwhy all these changes in classes19:11
sanujyes those changes are necessary19:11
@HeikoSCan't we do that in another patch?19:11
sanujbecause SGObject has a method called "set"19:11
sanujso compiler errors19:11
@HeikoSI see19:11
@HeikoSok I will stop moaning about them then19:11
@HeikoSsanuj: why is this needed?19:12
@HeikoSthe parameter combination?19:12
@HeikoSno set in there19:12
sanujHeikoS, oh, i had removed use namespace std from that file19:13
@HeikoSsanuj: why?19:13
sanuji can reverse these changes19:13
@HeikoSsanuj: yeah please only do changes that are necessary19:14
@HeikoSand send other ones in a separate patch19:14
@HeikoSmakes it easier to track down things19:14
sanujHeikoS, alright19:14
sanujyeah i'll reverse those19:14
@HeikoSsanuj: finished reviewing19:15
@HeikoSsanuj: can you make sure everything is unit test covered?19:15
@HeikoSand I mean really everything?19:15
@HeikoSall the code you added19:15
@HeikoSsince this is critical19:15
sanujHeikoS, can you stress on which parts are missing in the unit tests19:16
@HeikoSlisitsyn: sanuj, lambday one thing I would also like to see is how fast this stuff is. Can we run a little benchmark? Maybe even compare to the old parameter system?19:16
@HeikoSsanuj: you need you use a took to check this19:16
sanujHeikoS, class CTaxonomy : public CSGObject19:17
sanuj^^ necessary19:17
@HeikoSsanuj: ok then19:17
@HeikoScheck this19:17
@HeikoSgenerated a report of test coverage19:17
@HeikoSwe want 100% in all tag based codes19:17
@HeikoSsince it is so low level19:17
@HeikoSsanuj: you can run the same tool locally19:17
sanujHeikoS, agree19:17
@HeikoSah wait is that the right build19:18
@HeikoSsanuj: actually19:18
@HeikoSthis might contain it already19:18
@HeikoSah no19:19
sanujHeikoS, coveralls?19:19
@HeikoSit stopped19:19
@HeikoShave to ask wiking19:20
@HeikoSwiking: is there a way to check code coverage of a PR without having to run something?19:20
@HeikoSwiking: I want to make sure that sanuj 's tag patch is 100% covered19:20
@HeikoSusually, coveralls should do it, but it stopped updating, not sure why19:22
sanujHeikoS, the PR is against shogun/tags19:22
sanujand not shogun/develop19:22
@HeikoSsanuj: that should be ok19:22
@HeikoShas feature branches19:22
@HeikoSsanuj: you can also run a tool locally19:22
@HeikoSbut lets say what viktor says first19:22
@HeikoSwe should get that going again actually19:23
@HeikoSI think it is very easy to check this19:24
@HeikoSthis is it19:25
@HeikoSjust run shogun-unit-test binary with it19:25
@HeikoSand turn optimization off before that19:25
@HeikoSwell, just follow the readme19:25
sanujHeikoS, yes i will19:25
sanuji have a few doubts19:25
@HeikoSsanuj: let me know how it goes I will  be around for a bit so can help you19:25
sanujHeikoS, you want to only hide std::string demangledType() from SWIG?19:26
sanujand not the whole of any class?19:26
@HeikoSsanuj: ah yes, sorry19:28
@HeikoSI just made this one comment19:28
@HeikoS but basically saying, please hide everything from swig19:28
sanujHeikoS, can't hide any from swig19:28
@HeikoSSWIG just needs to see the get/set methods in CSGObject19:28
@HeikoSand of course the necessary type definitions19:28
@HeikoSuh that actually makes me think19:28
@HeikoSdo we need to change the typemaps?19:29
sanujHeikoS, btw this is a small piece of swig which works with tags in shogun
sanujHeikoS, i don't know much about swig typemaps19:29
@HeikoSwhen you say works with swig/shogun19:29
@HeikoScan you provide a listing that accesses things via SWIG and tags?19:30
@HeikoSan example?19:30
sanujyes i can do that19:30
sanujthis is another branch originating from my tags branch19:30
@HeikoScool show me19:30
@HeikoSthis will answer the question I had19:30
shogun-notifier-shogun: Ariane Paola Gomes :develop * 58628bc / examples/meta/generator/targets/cpp.json:
shogun-notifier-shogun: Make DenseDistance type available within cookbooks.19:31
shogun-notifier-shogun: Ariane Paola Gomes :develop * 73b6246 / examples/meta/generator/targets/cpp.json:
shogun-notifier-shogun: Added RealDenseDistance type.19:31
shogun-notifier-shogun: Heiko Strathmann :develop * 567ce21 / examples/meta/generator/targets/cpp.json:
shogun-notifier-shogun: Merge pull request #3258 from arianepaola/feature/cookbook_type_densedistance19:31
sanujHeikoS, you need an example of all the features in tags PR in swig right?19:31
shogun-notifier-shogun: Make RealDenseDistance type available within cookbooks.19:31
sanujlike in python?19:31
sanujHeikoS, i need you to review one more PR :D19:32
sanujmkl cookbook19:32
@HeikoSI am going through all PRs atm19:32
@HeikoSgetting there :)19:32
sanuji need to sleep now19:34
sanujwill update these tomorrow19:34
sanujHeikoS, will you be available this weekend?19:34
@HeikoSsanuj: probably a bit19:35
@HeikoSsanuj: ok19:35
@HeikoSsleep well19:35
@HeikoSarianepaola: there?19:35
sanujgoodnight :)19:35
arianepaolahey HeikoS19:35
@HeikoSarianepaola: so for the cookbooks19:35
@HeikoSthey also serve as integraiton test19:35
@HeikoSwhen you run make test locally19:35
@HeikoSevery cpp meta example outputs a serialized file19:36
@HeikoSwith all the results of the meta exampe19:36
@HeikoSwe then have an integration tester program that compares the output against a reference file19:36
-!- sanuj [~sanuj@] has quit [Quit: Leaving]19:36
@HeikoSso when I say the cookbook needs integration testing data, then I mean that you need to copy this file into your data repo and send a patch there19:36
@HeikoSthen update the data version in the cookbook pr19:37
@HeikoSall described in the readme as well19:37
@HeikoSyou can see whether it worked by checking the cpp travis build and see whether your new example is in the integration testing list19:37
@HeikoSarianepaola:  looks like this:
shogun-buildbotbuild #49 of xenial - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>19:38
@HeikoSfor it to be there, I first need to merge the data file19:38
@HeikoSbut as said, all in the readme19:38
arianepaolaI will add the integration test data19:39
@HeikoSarianepaola: ask me if you have troubles with this19:39
@HeikoSshould be really quic19:39
shogun-buildbotbuild #2908 of bsd1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Heiko Strathmann <>19:39
-!- travis-ci [] has joined #shogun19:41
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []19:41
arianepaolawiking and HeikoS: the module cannot be installed as normal module and needs to be injected into the package structure20:03
arianepaoladata_files=[(os.path.join('lib', glob.glob('install/lib/*/dist-packages')[0].split(os.sep)[-2], glob.glob('install/lib/*/dist-packages')[0].split(os.sep)[-1]), [glob.glob('install/lib/*/dist-packages/')[0]])]20:03
@HeikoSarianepaola: what does injected here mean?20:03
arianepaolaHeikoS: easy peasy20:03
@HeikoSpeasy easy ;)20:04
arianepaolaI will clean that line up20:04
arianepaolait is reconstructing from the lowest directory e.g. dist-packages going its way up pythonx.x -> lib and then putting it into the package at a lower directory (package is /usr/local)20:04
arianepaolaso it injects it at lib and python resolves itself then20:05
@HeikoSah man20:05
@HeikoSbut ok20:05
@HeikoSI think thats a fair solve20:05
arianepaolait will produce: /usr/local/lib/python2.7dist-packages with,, shogun and its submodules20:06
@HeikoSarianepaola: I guess that was the last blocker wasnt it=20:10
@HeikoSfor the python install thing20:11
arianepaolait seems so20:11
arianepaolathe cmake bootstrapping relies on distutils as that part is not possible to do with setuptools20:11
arianepaolaI will include the module injection into the gist, test it and start to include changes from the PR20:12
arianepaolahopefully without any distutils issues and will update the PR today20:12
shogun-buildbotbuild #877 of deb4 - python3 is complete: Failure [failed install test python modular]  Build details are at  blamelist: Ariane Paola Gomes <>20:14
shogun-buildbotbuild #50 of xenial - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>, Ariane Paola Gomes <>20:16
@HeikoSlooking forward to see that20:17
@HeikoSwiking: will be happy as well I guess :)20:17
@HeikoSI have to go now, but will check back tomorrow20:18
shogun-buildbotbuild #2909 of bsd1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Heiko Strathmann <>, Ariane Paola Gomes <>20:18
@HeikoSto merge things :)20:18
shogun-buildbotbuild #37 of FC23 - libshogun - aarch64 is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>20:18
shogun-buildbotbuild #465 of CentOS7 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>, Ariane Paola Gomes <>20:19
arianepaolathanks HeikoS20:20
@HeikoSarianepaola: see you!20:20
-!- HeikoS [] has quit [Quit: Leaving.]20:20
-!- travis-ci [] has joined #shogun20:23
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []20:23
shogun-buildbotbuild #878 of deb4 - python3 is complete: Success [build successful]  Build details are at
shogun-buildbotbuild #38 of FC23 - libshogun - aarch64 is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>, Ariane Paola Gomes <>20:57
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 272 seconds]21:29
arianepaolawiking and HeikoS, please check
-!- shogun-notifier- [] has quit [Quit: transmission timeout]22:31
-!- sonne|osx [] has joined #shogun23:18
-!- sonne|osx [] has quit [Quit: sonne|osx]23:46
--- Log closed Sat Jun 11 00:00:35 2016