Open in new window / Try shogun cloud
--- Log opened Mon Mar 18 00:00:14 2013
-!- zxtx [~zv@] has joined #shogun02:24
-!- zxtx [~zv@] has quit [Ping timeout: 240 seconds]02:43
-!- zxtx [~zv@] has joined #shogun03:14
-!- zxtx [~zv@] has quit [Ping timeout: 256 seconds]03:48
-!- zxtx [~zv@] has joined #shogun03:53
-!- sidi [~sidi@] has joined #shogun03:58
-!- sidi [~sidi@] has quit [Read error: Connection reset by peer]04:09
-!- zxtx [~zv@] has quit [Ping timeout: 264 seconds]04:27
-!- sidi [~sidi@] has joined #shogun04:53
-!- sidi [~sidi@] has quit [Client Quit]04:56
-!- sidi [~sidi@] has joined #shogun04:56
@sonney2kwiking, that would mean we always have merge conflicts in the file and this file would be committed on every attempt05:09
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]05:36
-!- zxtx [] has joined #shogun05:47
-!- sidi_ [~sidi@] has joined #shogun06:30
-!- sidi [~sidi@] has quit [Read error: Connection reset by peer]06:30
-!- hoijui [] has joined #shogun07:33
-!- n4nd0 [] has joined #shogun08:16
-!- zxtx [] has quit [Ping timeout: 240 seconds]08:20
-!- zxtx [] has joined #shogun08:22
-!- sonne|work [] has joined #shogun08:48
sonne|workwiking: heyja I have an idea how to get rid of the version.h file :-)09:05
sonne|workwe just do everything on-the-fly with defines on the cmdline!09:06
sonne|workwiking: then we could use the scheme we want :D09:06
sonne|workand for releases we just rely on proper tagging - i.e. 2.1 with time from NEWS09:09
-!- sidi_ [~sidi@] has quit [Read error: Connection reset by peer]09:31
-!- blackburn [] has joined #shogun09:48
wikingsonne|work: interesting idea10:02
wikingsonne|work: but i'm sure many opensource project has already solved this somehow10:02
sonne|workwiking: then please point me to anyone who solved this10:03
sonne|workblackburn: I had this idea how we could use git flow10:04
sonne|workblackburn: we would drop the lib/versionstring.h thing10:05
sonne|workand create things on the file (if .git dir exists)10:05
sonne|workand otherwise take the version info from NEWS file10:05
blackburnsonne|work: yeah makes sense for me10:05
sonne|workblackburn: so we would need to add lots of stuff to $DEFINE (all on cmdline) instead of in a .h file10:07
sonne|workthe only trouble is that anyone using libshogun would not be able to figure out the version10:08
blackburnlet me check what is defined there10:08
sonne|workbecause no such file would exist10:08
sonne|work(which sucks)10:08
blackburnsonne|work: so jsut extrac these defines from git?10:09
sonne|workblackburn: well we do that with .version.sh10:09
sonne|workohh better10:09
sonne|workwe generate lib/versionstring.h always on the fly10:09
sonne|workso we just need a small change to .version.sh10:09
blackburnsonne|work: sounds good10:10
-!- ptizoom [~christian@] has joined #shogun10:14
sonne|workblackburn: alright then we can follow from now on10:15
sonne|workptizoom: zoom zoom!10:15
ptizoomyep... still at ground NAN!10:15
ptizoomis shogun-toolbox surviving?10:16
blackburnptizoom: interesting question10:17
ptizoomwell...I overlooked lots of # packages...10:18
ptizoomptizoom is a strange name, and not very flattering...but unique. never used by anyone since ~1998!10:22
blackburnptizoom: yeah I chose a name for my library using the same way10:23
wikingsonne|work: yeah i'm just looking10:29
sonne|workptizoom: what does ptizoom stand for?10:30
wikingsonne|work: btw why don't we just use the good old epoch.major.minor_revision format?10:34
sonne|workwiking: we do10:35
sonne|workshogun 2.1.010:35
wikingsonne|work: in the header as well?10:35
ptizoomklein zoom...!10:35
wikingsonne|work: i see the MAINVERSION macro10:36
wikingsonne|work: but we could switch to the usual slicing up it into different parts10:36
wikinglike most of the opensource project does it10:36
wikingdo we really need to know the git revision ?10:37
sonne|workwiking: sure10:37
sonne|workif someone reports an issue and has *some* git compiled version10:37
sonne|workhow do we know the version?10:37
wikingyou mean he would have a randomly checked out shogun version10:38
wikingand then he wouldn't be able to tell us which rev he used from the repo?10:38
sonne|workthat's the most common way of using shogun I guess10:38
wikingi mean without the header10:38
wikingsince afaik if they have a random version10:39
wikingthen they checked out the git repo10:39
wikinghence they can tell us which is the last git commit in their repo10:39
sonne|workwiking: if they didn't update in the meantime and only have a single version lying around yes10:39
wikingyes but i don't see it happening that somebody has *some* commit inbetween 2 releases, but he wouldn't be able to tell us his last commit w/o the version string10:40
wikingi mean how did he get that source in the first place w/o checking out the repo10:40
sonne|workwiking: I had that a lot10:40
wikingsonne|work: using then ?10:44
-!- sidi [~sidi@] has joined #shogun10:44
wikingand generate versionstring.h from that with the help of ./configure10:44
sonne|workwiking: I don't really understand what you are saying but we have a nice fix now so I guess that is good enough to no longer worry about this issue10:46
wikingwell the most common things that oss project do is simply have a .in file (you know the template file). i think that's the common format of autoconf10:47
wikingand then just use that to generate with ./configure script the actual .h file10:48
wikingpretty common stuff...10:48
sonne|workwiking: that is what we do10:48
sonne|workthough not with .in but with a shell script10:48
-!- n4nd0 [] has quit [Ping timeout: 264 seconds]11:09
-!- sidi [~sidi@] has quit [Read error: Connection reset by peer]11:24
-!- sidi [~sidi@] has joined #shogun11:41
-!- ptizoom [~christian@] has quit [Quit: Ex-Chat]12:55
-!- n4nd0 [] has joined #shogun13:43
-!- n4nd0 [] has quit [Client Quit]13:44
-!- n4nd0_ [] has joined #shogun13:44
-!- n4nd0_ is now known as n4nd013:44
-!- sidi [~sidi@] has quit [Read error: Connection reset by peer]14:43
-!- sidi [~sidi@] has joined #shogun14:45
-!- hoijui [] has quit [Ping timeout: 246 seconds]14:49
-!- blackburn [] has quit [Quit: Leaving.]15:00
-!- hoijui [~hoijui@] has joined #shogun15:46
-!- hoijui [~hoijui@] has quit [Ping timeout: 245 seconds]17:17
-!- blackburn [~blackburn@] has joined #shogun17:37
-!- sidi [~sidi@] has quit [Read error: Connection reset by peer]17:47
-!- sidi [~sidi@] has joined #shogun17:48
-!- sidi_ [~sidi@] has joined #shogun18:59
-!- sidi [~sidi@] has quit [Read error: Connection reset by peer]18:59
-!- sidi_ [~sidi@] has quit [Client Quit]19:01
-!- shogun-notifier- [] has joined #shogun19:06
shogun-notifier-shogun: Soeren Sonnenburg :master * 1e05037 / Makefile:
shogun-notifier-shogun: fix release script19:06
shogun-notifier-shogun: Soeren Sonnenburg :dev * 6ce4431 / src/ChangeLog:
shogun-notifier-shogun: updated changelog19:06
shogun-notifier-shogun: Soeren Sonnenburg :dev * 1b1130d / doc/Doxyfile:
shogun-notifier-shogun: Preparing for new Release shogun_2.1.019:06
shogun-notifier-shogun: Soeren Sonnenburg :dev * e6138dd / src/ChangeLog:
shogun-notifier-shogun: Preparing for new Release shogun_2.1.019:06
blackburnsonney2k: release every day!19:12
-!- hoijui [] has joined #shogun19:43
shogun-buildbot_build #364 of rpm1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>20:23
-!- zxtx [] has quit [Ping timeout: 264 seconds]20:28
wikinganybody around?20:30
blackburnwiking: yes20:30
wikingblackburn: so actually now with the version string changes, are we going for direct commit ?:)20:31
blackburnwiking: I don't mind direct commit20:32
wikingplease let's do it!20:32
blackburnwiking: I do direct commit anyway :D20:32
wikingsince if we really go forward with the git-flow-like dev mechanizm, we should not encounter merging problems20:32
-!- alexlovesdata [~binder@2001:638:806:e001:eda1:5a75:ff2a:e6f] has joined #shogun20:36
wikingbtw: fetching the upstream repo via 3g... isn't having the most optimal speed20:36
@sonney2kalexlovesdata, hey20:36
wikingblackburn: i've created a mini patch for ./configure20:36
alexlovesdataint32_t num_samples=features->get_num_vectors();20:36
alexlovesdataSGMatrix<float64_t> outmatrix(dim_feature_space,num_samples);20:36
alexlovesdatafloat64_t val = CMath::sqrt(2.0 / dim_feature_space);20:36
alexlovesdatafor (int32_t vecind = 0; vecind < num_samples; vecind++)20:36
alexlovesdatafor (int32_t od = 0; od < dim_feature_space; ++od)20:36
alexlovesdatafloat64_t tmpval=randomcoeff_additive[od]+features->dense_dot (vecind, randomcoeff_multiplicative+od*dim_input_space, dim_input_space);20:36
wikingfor jemalloc and tc malloc20:36
alexlovesdataCDenseFeatures<float64_t> * result = new CDenseFeatures<float64_t>(outmatrix);20:36
wikinglol face :D20:36
blackburnuh that's a rude hello20:37
wikingso yeah i have a tcmalloc,jemalloc patch for shogun..20:37
blackburnwiking: I don't give a what tcmalloc is :)20:37
wikingbasically a faster heap manager library than the default glibc20:37
wikingas well as both support heap profiling20:37
wikingand heap checking20:37
wiking-> leak checking20:37
wikingboth are developed by the big boyz20:38
alexlovesdatacode spam-in :P20:38
wikingjemalloc = facebook; tcmalloc = google20:38
blackburnwiking: with proper define would be nice20:38
wikingblackburn: ?20:39
blackburnwiking: not the default thing I mean20:39
wikingit's a configure flag20:39
wikingand by default it's disabled20:39
blackburnwiking: is it possible to change default allocator for everything?20:40
blackburnI don't know a way but may be you know20:40
wikingas it's a drop-in malloc replacement library, it doesn't need any extra stuff just to link with the given library20:40
wikingeven LD_PRELOAD should work....20:40
wikingaccording to tcmalloc you just do LD_PRELOAD20:40
blackburnwiking: just link?20:40
wikingand the malloc calls are going to be overriden20:40
wikingblackburn: yep... so when linking the shogun lib: -ltcmalloc20:41
wikingand that's all20:41
blackburnwiking: I am curious if there is a speedup20:41
wikingyeah i didn't manage to test that20:41
wikingi've created tcmalloc replacement yesterday night20:41
wikingbut then it came out that it's buggy under 10.8 osx20:41
-!- n4nd0 [] has quit [Quit: leaving]20:42
wikingso i quickly written the jemalloc support as well20:42
wikingas that works fine20:42
wikingbut didn't have time for debugging20:42
wikingbut since i've created we might as well put it in20:42
wikingas maaaybe somebody would like to play with it20:42
wikingand then if somebody has time to test it....20:42
wikingi mean it shoudl be fairly easy to check this20:42
wikingthe bsd1 bot uses jemalloc fork by default (freebsd's malloc library isn't based on glibc)20:43
wikingso we could check there running times for the examples20:43
blackburnwiking: any package in ubuntu?20:43
wikingblackburn: dunno just a sec20:43
wikinglibtcmalloc-minimal0 - an efficient thread-caching malloc20:43
wikinglibtcmalloc-minimal0-dbg - an efficient thread-caching malloc20:43
wikinglibjemalloc-dev - development files and documentation for jemalloc20:44
wikinglibjemalloc1 - general-purpose scalable concurrent malloc(3) implementation20:44
wikingso yes20:44
blackburnlet me try it with tapkee20:44
wikingboth are in20:44
blackburnI am curious20:44
wikingblackburn: LD_PRELOAD it20:44
blackburnwiking: I feel stupid - what does it mean?20:44
wikinge.g. LD_PRELOAD="/usr/lib/"20:45
blackburnwiking: and run app?20:45
wikingLD_PRELOAD="/usr/lib/" ./example20:45
blackburnwiking: no need for any linking you mean?20:45
wikingblackburn: shoudl work w/o anything20:45
wikingthey don't recommend this mode20:45
wikingbut should work20:45
wikingas it really overrides the *malloc() functions20:45
wikingand as it preloads that lib20:46
wikinganybody who calls *malloc will end up calling tcmalloc's functions20:46
wikingbut yeah otherwise just append a -ltcmalloc to your linking flags20:46
blackburnwiking: no significant speedup on a toy example20:46
wikingbtw: do you have memory heavy examples?20:47
blackburnwiking: probably I do not malloc often20:47
blackburnwiking: no, but I was curious still20:47
blackburnwiking: with -O3 it could avoid any mallocs at all hah20:47
wikingsee the graphs on the bottom of the page20:47
blackburnwiking: it is a good thing for servers I guess20:47
wikingmmm well anything that needs to be scalled ;)20:48
wiking*scaled :)20:48
wikinganyhow it's not only this20:48
wikingheap-profiling is really handy as well20:48
wikingbut with the tcmalloc package in ubuntu you cannot do it20:49
wikingas it is the _minimal version20:49
wikingthat doesn't have any profiling support20:49
blackburnwiking: I see20:49
wikingheiko ? or he gut exhausted from the bughunt... ? :S20:49
blackburnwiking: btw I solved gtest issue20:54
wikingblackburn: u had one? :)20:54
blackburnwiking: well yes20:54
wikingwith tapkee u mean/20:54
blackburnwiking: I am playing with tapkee to learn new things :D20:55
wikingcool cool20:55
wikingthe clang-analyzer output is really getting huge20:56
wikingit's 3gigs now20:56
wikingand only having february as tar.bz220:56
blackburnwiking: I failed to set up analyzer and tapkee :D20:56
wikingand march non archived20:56
blackburnno output at all..20:56
blackburnwiking: and about t-SNE20:57
blackburnwiking: it is ready and I even play with it but I still wait for laurens20:57
blackburnhe said he don't mind gpling but disappeared then20:57
blackburnit would be faster to write it once again hah20:57
wikingwtf. freebsd has clang but i cannot find scan-build20:58
blackburnwiking: so for shogun no tricks with scan-build were made?20:59
wikingno not really21:00
wikingthe basic commands that are in the examples21:00
wikingbut i guess it's a bit different with cmake21:00
blackburnwiking: well I tried with exact g++ calling command21:01
blackburngot nothing again21:01
wikingbtw i think the scan-build in ubuntu lts is still not the best21:01
wikingas it reports a lot of stupid things....21:01
wikingas if it doesn't fail on assertation errors21:01
blackburnI should update my system21:01
wikinglike this one21:02
wikingi dont understand why is it not realising tht there's and assert on rn121:02
wikingand this doesn't appear on my scan-build of shogun21:03
wikingthat one realises that actually the assert catches the null pointer dereferencing21:03
@sonney2kblackburn, look google is accepting applications now!21:10
alexlovesdatawiking,blackburn: I did not got what you talked :D , I will log off, have a nice evening21:11
-!- alexlovesdata [~binder@2001:638:806:e001:eda1:5a75:ff2a:e6f] has left #shogun []21:11
blackburnsonney2k: yeah application is improving, we will send it in time21:11
blackburnsonney2k: it is ok to send a day before deadline right?21:11
blackburnsonney2k: the application itself is quite easy but we have to detalize ideas21:13
wikingblackburn: plz approve my request21:15
blackburnwiking: sure21:15
blackburnwiking: I shared with wiking <at> before21:15
wikingyep unforunately it's not google accounted21:15
@sonney2kblackburn, no it is not easy and detailizing ideas is sth we should try hard now21:18
@sonney2kblackburn, wiking I have another git flow blocker21:19
blackburnsonney2k: detalizing ideas is much harder for me21:19
@sonney2kwe currenlty have DOXYGEN 's version also encoded in doc/Doxyfile21:19
wikingsonney2k: listeninng.... it's almost as if u find cockblockers in shogun ;)21:19
@sonney2kblackburn, well draft *1* now21:19
@sonney2kblackburn, then ask all the mentors now21:20
blackburnsonney2k: asked already21:20
wikingsonney2k: you mean: PROJECT_NUMBER         = v2.1.021:20
blackburnsonney2k: twos21:20
@sonney2kwiking, yes21:20
@sonney2kblackburn, whom did you ask?21:20
wikingsonney2k: that's easy to solve21:20
blackburnsonney2k: igor and cheng21:20
wikingsonney2k: that's a common way of having doxyfile.in21:21
wikingsonney2k: this is how i solved this in another project of mine:;a=blob;f=doc/;h=fea318f77412097f06f52c1481eb6c6e3b678af5;hb=HEAD21:21
@sonney2kwiking, so we generate Doxyfile from ?21:21
wikingsonney2k: yes21:22
wikingafaik should be done as well ;)21:24
wikingbut yeah i guess i told this already21:24
shogun-notifier-shogun: Soeren Sonnenburg :master * d946b1d / doc/Makefile,doc/,doc/
shogun-notifier-shogun: always version doxygen files based on NEWS21:34
shogun-notifier-shogun: Soeren Sonnenburg :master * 68f6a48 / Makefile,src/,src/Makefile.template:
shogun-notifier-shogun: autogenerate lib/versionstring.h based on NEWS file if no .git repo is available21:34
@sonney2kblackburn, once you have a draft for one task on the website I would send it to the other usual suspects (olivier, quok, alex, vojtech, ...)21:35
blackburnsonney2k: tomorrow's morning21:36
shogun-buildbot_build #1003 of deb1 - libshogun is complete: Failure [failed compile]  Build details are at  blamelist: Soeren Sonnenburg <>21:38
-!- travis-ci [] has joined #shogun21:39
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []21:39
shogun-notifier-shogun: Soeren Sonnenburg :master * a482789 / src/
shogun-notifier-shogun: add 0xMMmmrr (MM==major mm==minor rr=release) to version22:05
-!- travis-ci [] has joined #shogun22:15
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []22:15
shogun-buildbot_build #1004 of deb1 - libshogun is complete: Success [build successful]  Build details are at
-!- zxtx [zv@conference/pycon/x-pscdziifemaqmrdh] has joined #shogun22:27
-!- zxtx [zv@conference/pycon/x-pscdziifemaqmrdh] has quit [Ping timeout: 255 seconds]22:54
-!- hoijui [] has quit [Ping timeout: 264 seconds]22:56
shogun-buildbot_build #365 of rpm1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>23:05
-!- zxtx [zv@conference/pycon/x-rnauhrcjrxankcvr] has joined #shogun23:18
--- Log closed Tue Mar 19 00:00:14 2013