Open in new window / Try shogun cloud
--- Log opened Sat Jul 14 00:00:17 2012
-!- blackburn [~blackburn@] has left #shogun []00:21
-!- gsomix [~gsomix@] has quit [Ping timeout: 246 seconds]00:31
-!- puffin444 [62e3926e@gateway/web/freenode/ip.] has quit [Quit: Page closed]00:44
-!- n4nd0 [] has quit [Ping timeout: 265 seconds]00:51
-!- n4nd0 [] has joined #shogun01:18
-!- emrecelikten [~emre@] has quit [Ping timeout: 248 seconds]01:23
-!- n4nd0 [] has quit [Quit: leaving]02:23
shogun-buildbotbuild #22 of nightly_none is complete: Failure [failed compile]  Build details are at
shogun-buildbotbuild #28 of nightly_default is complete: Failure [failed compile]  Build details are at
shogun-buildbotbuild #18 of nightly_all is complete: Failure [failed compile]  Build details are at
-!- pluskid [~pluskid@] has joined #shogun06:40
-!- romi_ [~mizobe@] has quit [Remote host closed the connection]06:48
-!- pluskid [~pluskid@] has quit [Quit: Leaving]06:52
-!- zxtx [] has joined #shogun07:18
CIA-18shogun: Evgeniy Andreev master * rd450473 / (3 files in 2 dirs): fixed warnings and bugs -
CIA-18shogun: Evgeniy Andreev master * rcc22b2c / (2 files): added buffer protocol example -
CIA-18shogun: Soeren Sonnenburg master * r5068d3e / (5 files in 3 dirs): Merge pull request #646 from gsomix/buffer_protocol -
shogun-buildbotbuild #139 of deb3 - modular_interfaces is complete: Failure [failed compile ruby_modular]  Build details are at  blamelist: Evgeniy Andreev <>08:20
shogun-buildbotbuild #140 of deb3 - modular_interfaces is complete: Failure [failed compile ruby_modular]  Build details are at  blamelist: Evgeniy Andreev <>, Soeren Sonnenburg <>08:31
-!- gsomix [~gsomix@] has joined #shogun09:01
gsomixgood morning09:02
gsomixsonney2k, around?09:02
-!- n4nd0 [] has joined #shogun11:37
-!- bern4rd [4f940120@gateway/web/freenode/ip.] has joined #shogun11:41
-!- blackburn [~blackburn@] has joined #shogun11:52
n4nd0blackburn: I mentioned you yesterday in my blog :)11:54
blackburnn4nd0: cool, what did I do? ;)11:55
n4nd0blackburn: you helped me to find the memory leaks11:55
n4nd0with the GCDEBUG suggestion ;)11:55
-!- bern4rd [4f940120@gateway/web/freenode/ip.] has quit [Quit: Page closed]12:26
CIA-18shogun: Sergey Lisitsyn master * r3523a55 / (3 files): Get rid of wrong SGReferencedData based assignment when using CMap -
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]12:39
-!- blackburn [~blackburn@] has joined #shogun12:40
shogun-buildbotbuild #141 of deb3 - modular_interfaces is complete: Failure [failed compile ruby_modular]  Build details are at  blamelist: Sergey Lisitsyn <>12:45
CIA-18shogun: Evgeniy Andreev master * rccb9c80 / (4 files in 2 dirs): fixed compile error, buffer stuff now is in typemaps -
CIA-18shogun: Sergey Lisitsyn master * rb3303c6 / (4 files in 2 dirs): Merge pull request #647 from gsomix/buffer_protocol -
shogun-buildbotbuild #39 of bsd1 - libshogun is complete: Failure [failed compile]  Build details are at  blamelist: Sergey Lisitsyn <>13:34
n4nd0blackburn: around?13:37
n4nd0does this look weird to you?
n4nd0it segfaults13:38
n4nd0I don't really understand why13:38
blackburnyou need to do13:38
n4nd0what is it?13:38
blackburnfor (int32_t i=0; i<new_num_cols; i++) new (&out[i]) SGMatrix<T>();13:39
blackburnbefore setting matrices13:39
blackburnn4nd0: you allocate memory for matrices but reference counts are not allocated13:39
blackburnand when you try to do =13:39
blackburnit tries to free up matrix created before13:40
blackburnhaving no reference count allocated at all it kabooms13:40
n4nd0but could the operator check if there is something allocated beforing trying to free up?13:40
n4nd0the operator= I mean13:41
n4nd0beforing = before ^ ....13:41
blackburnI am not sure it would work anyway13:42
n4nd0blackburn: do you mean with the new (&out[i]) SGMatrix<T>();?13:42
blackburnno, some check in =13:42
n4nd0what does new (&out[i]) SGMatrix<T>() exactly do anyway?13:43
blackburninplace new13:44
n4nd0I don't think I get what the part (&out[i]) does13:44
blackburnit is a pointer you use13:44
n4nd0inplace? ok I am going to read about it :)13:44
blackburnit acts just like 'new' operator but exactly to &out[i] and not to some new piece of memory13:44
n4nd0I see13:45
blackburnin normal way it allocates object somewhere and returns a pointer13:45
blackburnbut here we control where to place a pointer13:45
blackburnnot pointer object I mean13:45
shogun-buildbotbuild #142 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: Evgeniy Andreev <>13:56
-!- n4nd0 [] has quit [Ping timeout: 246 seconds]13:57
blackburnsonney2k: I want to cut all the optimization abilities outdated stuff in configure14:07
blackburnit is really outdated - just noticed it forces to use nocona here - my cpu is corei7-avx14:07
shogun-buildbotbuild #143 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: Sergey Lisitsyn <>14:25
-!- n4nd0 [] has joined #shogun14:37
n4nd0blackburn: I have read that after using an inplace constructor one shouldn't call delete to free the memory used but the destructor directly14:48
n4nd0blackburn: do you recognize that?14:48
blackburnn4nd0: yeah14:48
n4nd0so no SG_UNREF14:48
blackburnyeah forgot to say that14:54
blackburnn4nd0: well actually we kinda emulate new[] and delete[]15:01
n4nd0blackburn: delete with SG_UNREF but new?15:01
blackburnn4nd0: no I mean SG_MALLOC + inplace new = new []15:01
blackburnand explicitly call dtors and SG_FREE is delete []15:02
n4nd0blackburn: so I could just use new[]?15:02
blackburnn4nd0: yeah may be :D15:02
blackburnnew[] is not tracked unfortunately15:03
n4nd0blackburn: something line new SGMatrix<T>(num_rows, num_cols)[num]15:03
blackburnyeah that should work too15:03
n4nd0what do you mean with that it is not tracked?15:04
blackburnsg_trace_mallocs thing won't work15:04
n4nd0I don't follow you15:05
blackburnn4nd0: we have --enable-trace-mallocs option for configure15:05
blackburnthat stores all SG_MALLOCs in map15:05
blackburnand if something is not freed on exit15:05
blackburnit will output15:06
n4nd0I see15:06
n4nd0this new SGMatrix<T>(num_rows, new_num_cols)[num_components] didn't make the trick15:09
n4nd0compile error15:09
blackburnthen use that trick :D15:12
n4nd0which one?15:12
blackburnSG_MALLOC + inplace new + explicit dtors + SG_FREE15:13
-!- blackburn [~blackburn@] has left #shogun []15:35
-!- gsomix [~gsomix@] has quit [Quit: Ex-Chat]16:52
-!- gsomix [~gsomix@] has joined #shogun16:52
-!- puffin444 [62e3926e@gateway/web/freenode/ip.] has joined #shogun18:05
-!- bern4rd [4f940120@gateway/web/freenode/ip.] has joined #shogun18:33
-!- alexlovesdata [] has joined #shogun18:39
n4nd0hey guys18:46
n4nd0I just updated master and I am getting18:46
n4nd0modshogun_wrap.cxx.o: In function `LongIntFeatures_getbuffer':18:46
n4nd0/home/nando/Documents/shogun/shogun/src/interfaces/python_modular/modshogun_wrap.cxx:339264: undefined reference to `shogun::CDenseFeatures<long>::get_feature_matrix(int&, int&)'18:46
n4nd0modshogun_wrap.cxx.o: In function `ULongIntFeatures_getbuffer':18:46
n4nd0/home/nando/Documents/shogun/shogun/src/interfaces/python_modular/modshogun_wrap.cxx:340564: undefined reference to `shogun::CDenseFeatures<unsigned long>::get_feature_matrix(int&, int&)'18:46
n4nd0is it something local?18:47
gsomixn4nd0, strange18:55
gsomixit's my code18:56
n4nd0gsomix: aham, do you get it too locally?18:56
gsomixn4nd0, "locally". what do you mean? this stuff is in swig_typemaps.i19:00
gsomixn4nd0, try make clean, hm19:00
n4nd0yeah I did ig19:11
n4nd0I meant locally in your machine19:12
n4nd0gsomix: I don't think this is happening in the buildbot since python_modular compiles19:13
n4nd0my version of swig is SWIG Version 2.0.419:13
n4nd0gsomix: can that be a problem?19:13
gsomixn4nd0, I use swig 2.0.4 so.19:14
gsomixn4nd0, may be a problem in the order of include files. I need to check.19:17
n4nd0gsomix: let me know if you find something please19:17
-!- blackburn [~blackburn@] has joined #shogun19:29
gsomixn4nd0, it seems that it's all right.19:30
-!- gsomix [~gsomix@] has quit [Quit: Ex-Chat]19:34
-!- gsomix [~gsomix@] has joined #shogun19:34
n4nd0gsomix: mmm ok19:38
n4nd0I have run make clean, configure and I am compiling again19:39
n4nd0let's see19:39
gsomixn4nd0, ?19:43
n4nd0still failing19:50
n4nd0blackburn: any idea?19:51
n4nd0modshogun_wrap.cxx.o: In function `LongIntFeatures_getbuffer':19:51
n4nd0/home/nando/Documents/shogun/shogun/src/interfaces/python_modular/modshogun_wrap.cxx:339259: undefined reference to `shogun::CDenseFeatures<long>::get_feature_matrix(int&, int&)'19:51
n4nd0modshogun_wrap.cxx.o: In function `ULongIntFeatures_getbuffer':19:51
n4nd0/home/nando/Documents/shogun/shogun/src/interfaces/python_modular/modshogun_wrap.cxx:340559: undefined reference to `shogun::CDenseFeatures<unsigned long>::get_feature_matrix(int&, int&)'19:51
gsomixn4nd0, only Long and ULong features?19:51
n4nd0it seems so19:51
n4nd0gsomix: does it give any clue?19:52
@sonney2kblackburn, grab the latest sources from mplayer and merge the changes to detect your cpu in configure  - the stuff is not really outdated!19:55
gsomixsonney2k, \(^_^)/19:57
@sonney2kgsomix, can you just add a try except NameError around your memoryview expression to fix the buffer proto example?19:57
@sonney2kgsomix, ^ that is stil failing19:58
@sonney2kgsomix, hi there19:58
gsomixsonney2k, yeah, memoryview introduced only in python2.719:58
gsomixI'll fix it, one moment19:59
n4nd0sonney2k: hi, do you happen to know why can this be caused?
n4nd0it looks that it compiles ok in the buildbot but not in my machine20:00
@sonney2kn4nd0, also compiles fine here btw20:03
-!- blackburn [~blackburn@] has quit [Read error: Connection reset by peer]20:03
@sonney2kn4nd0, maybe one test before we dig deeper20:03
-!- blackburn1 [~blackburn@] has joined #shogun20:03
@sonney2kn4nd0, do git clean -dfx (but make sure you have no uncommitted stuff)20:04
@sonney2kthen recompile20:04
n4nd0untracked files are no problem right?20:04
n4nd0sonney2k: they won't disappear, will they?20:04
@sonney2kn4nd0, this error cannot really happen except if you do ./configure --datatypes= ... and then add some custom stuff there20:05
@sonney2kn4nd0, they disappear20:05
@sonney2keverything that is not tracked will be gone20:05
-!- bern4rd_ [4f940120@gateway/web/freenode/ip.] has joined #shogun20:06
-!- bern4rd_ [4f940120@gateway/web/freenode/ip.] has quit [Client Quit]20:06
blackburn1(09:55:50 PM) sonney2k: blackburn, grab the latest sources from mplayer and merge the changes to detect your cpu in configure  - the stuff is not really outdated! --------- why to detect it?20:06
n4nd0sonney2k: they didn't disappear ;)20:06
@sonney2kblackburn1, because on some archs / machines it helps to get faster code20:07
blackburn1sonney2k: native goes to correct march on my machine20:07
@sonney2kn4nd0, are these in .gitignore perhaps?20:07
n4nd0sonney2k: no, otherwise they won't appear as untracked when doing git status20:07
@sonney2ktouch shogun/lib/bla.cpp20:08
@sonney2kgit status20:09
@sonney2k# Untracked files:20:09
@sonney2kgit clean -dfx20:09
@sonney2kRemoving shogun/lib/bla.cpp20:09
@sonney2kn4nd0, anyways - compile and let us know if the error appears again20:10
n4nd0sonney2k: it didn't remove mines under examples/undocumented/libshogun and python_modular20:10
blackburn1sonney2k: echo 'int main(){return 0;}' > test.c && gcc -v -Q -march=native -O3 test.c -o test && rm test.c test20:10
n4nd0sonney2k: ok, copilation on the way right now20:10
blackburn1n4nd0: could you please do ^ too?20:10
blackburn1and check -march parameter20:10
n4nd0blackburn1: what do you meaan with check?20:11
blackburn1n4nd0: in output there will be -march=something20:11
n4nd0 -march=core2 -mcx16 -msahf -mno-movbe -mno-aes -mno-pclmul -mno-popcnt20:12
blackburn1aham thanks20:12
-!- blackburn1 is now known as blackburn20:12
blackburnsonney2k: I am pretty sure native detects right march20:12
n4nd0sonney2k: still, compilation error20:13
n4nd0sonney2k: it didn't remove those files because I was not in a root directory of their path20:19
n4nd0sonney2k: they were in shogun/examples/undocumented/...20:19
n4nd0sonney2k: I was in shogun/src20:19
n4nd0now doing git clean -dfx from shogun/ removed them20:19
gsomixsonney2k, fixed, I also removed ctypes from  buffer protocol code.20:24
n4nd0bah, the error is still there :(20:27
blackburndoes anybody here use guake?20:31
n4nd0blackburn: I installed it when you told me about it, but I don't use it much to tell the truth20:32
@sonney2kblackburn, march=core220:32
n4nd0I am loyal to my terminator :)20:32
blackburnsonney2k: wrong?20:32
@sonney2kn4nd0, I don't get why it happens20:32
CIA-18shogun: Evgeniy Andreev master * r4ef2573 / (3 files in 3 dirs): minor fixes for buffer protocol -
CIA-18shogun: Sergey Lisitsyn master * r1ab1290 / (3 files in 3 dirs): Merge pull request #648 from gsomix/buffer_protocol -
gsomixn4nd0, try again20:32
blackburnn4nd0: I patched it today to display current dir in tab name20:33
@sonney2kblackburn, no correct20:33
n4nd0blackburn: nice20:33
n4nd0gsomix: let's see20:33
blackburnsonney2k: so currently march=native produces correct arch for you, me, gsomix and n4nd020:34
@sonney2kblackburn, sure20:34
blackburnI am pretty sure it would work for anything else20:34
blackburnwhy to do that then?20:34
@sonney2kblackburn, not all gccs have that20:34
blackburnsonney2k: well I see no reason to use <4.3 now20:35
blackburnwe force to use latest swig anyway20:35
gsomixsonney2k, btw, I'm working today on arithmetic operators stuff. it's works for all classes with buffer.20:36
gsomixDenseFeatures+=<class with buffer>, I mean20:36
blackburnis it possible to DenseFeatures+=SparseFeatures?20:37
gsomixtrying to hacks it in shogun20:37
gsomixyep, if SparseFeatures will return as a matrix of values in buffer20:38
-!- bern4rd [4f940120@gateway/web/freenode/ip.] has quit [Quit: Page closed]20:39
-!- gsomix [~gsomix@] has quit [Quit: Ex-Chat]20:41
-!- gsomix [~gsomix@] has joined #shogun20:41
n4nd0good gsomix! you fixed it :)20:44
n4nd0gsomix, blackburn, sonney2k problem solved!20:44
n4nd0gsomix: what was it then?20:44
n4nd0python version?20:44
gsomixn4nd0, I dunno lol. I just removed ctype (like unsigned long), because we have int64...20:45
shogun-buildbotbuild #40 of bsd1 - libshogun is complete: Success [build successful]  Build details are at
n4nd0gsomix: haha ok20:45
-!- heiko [] has joined #shogun20:45
puffin444I have found a very serious bug in the model selection framework20:46
heikopuffin44, oh no, what is it?20:48
puffin444When I register a SGVector as a parameter, it has a pointer to the underlying SGVector.vector.20:48
puffin444The problem is that if the SGVector is ever manipulated (i.e. resize) or reassigned the parameter still has the pointer to the original SGVector.vector.20:48
heikothats why usually a pointer to the pointer is stored20:50
puffin444So if I ever try to resize a SGVector in an object, I can never access it through the model selection framework. Even worse, I think this leads to a memory leak.20:50
heikoso that if the data changes, the pointer still is good20:50
heikopuffin444, Ill have a look tomorrow, no time today unfortunately. sorry20:51
puffin444Ok. I will send an example of what I mean over the mailing list.20:52
@sonney2kgsomix, blackburn - buffer protocol is just for dense matrices. so we could in principle support it for sparse matrices too but that would mean that we have to expand to dense first...20:52
gsomixas I said20:53
@sonney2kgsomix, yes the fix is correct20:56
@sonney2kblackburn, I would rather stick to what the mplayer guys do20:57
blackburnsonney2k: they have outdated as well20:59
blackburnno sse4 check, no i7 support20:59
blackburnthe latest thing they did is core220:59
gsomixsonney2k, btw, where are you now?21:00
@sonney2kgsomix, do you want to send a missile?21:00
blackburnsonney2k: I know you are a big fan of handmade and DIY but I see no actual reason to have such crappy config no one supports21:01
@sonney2kblackburn, I think they check for gcc version and if it is new enough fall back to march=native21:01
@sonney2kblackburn, so please check if it does21:01
gsomixsonney2k, huh, only dumplings :)21:01
@sonney2kgsomix, it would totally suffice if you stay around after gsoc21:02
@sonney2kI am pretty happy with what you do for shogun :)21:02
blackburnthey have basically the same thing we do21:02
@sonney2kblackburn, please test run their configure21:03
@sonney2kI would expect it to find march=native on your cpu just fine21:03
blackburnsonney2k: yeah native21:06
gsomixsonney2k, no problem. I'm happy too, there is ?ool team. and thanks (*^__^*)21:06
blackburngsomix: you must be happy to hear it, sonney2k asks me constantly to leave team21:07
blackburnhe even suggested some money21:07
gsomixblackburn, huh, how many? ;)21:08
@sonney2kblackburn, what?21:10
blackburngsomix: he denies me to tell :(21:10
gsomixblackburn, btw you should 'prostavitsya'.21:11
blackburnwhy should I21:11
shogun-buildbotbuild #144 of deb3 - modular_interfaces is complete: Success [build successful]  Build details are at
gsomixblackburn, you didn't 'prostavitsya' last year after gsoc.21:12
blackburnI found that silly21:13
gsomixokay :(21:14
gsomixtime to work21:15
gsomixsonney2k, overloading [] in c++ have no effect. it's needed overloading by hands.21:18
gsomixin swig code I mean21:19
blackburnsaturday night is time to party!21:19
blackburnor to play WoW, no idea what normal people do21:19
gsomixblackburn, dota221:19
-!- blackburn is now known as never_tell_me_ab21:20
gsomixfree to play now, just in steam :)21:20
-!- never_tell_me_ab is now known as blackburn21:20
gsomixblackburn, lol as lol, not LeagueOfLegends :)21:22
n4nd0gsomix, blackburn do you play games guys?21:23
blackburnI fscking hate games and gamers21:24
@sonney2kgsomix, actually it should enable us to do sth like x+=y (when x and y are e.g. RealFeatures)21:24
gsomixn4nd0, huh, no :)21:24
@sonney2kblackburn, on some kind of holy war today?21:24
blackburnsonney2k: why so?21:24
gsomixn4nd0, only small indie-games.21:24
blackburnI hate optics and games all the way21:24
n4nd0gsomix: no idea what tahs is :P21:25
@sonney2kgsomix, but right we don't have a typemap for numpy -> SimpleFeautres so I guess your way is what we need21:25
gsomixlike Angry Birds on my IPad^W21:25
puffin444heiko, don't worry about the bug. It's something I screwed up.21:26
n4nd0gsomix: aham, ok :)21:26
gsomixsonney2k, mmm, I just doing it with using numpy.array's buffer.21:26
@sonney2kgsomix, what is ?????????????21:26
blackburnsonney2k: to get some beer/vodka/anything in case of some achievement21:26
blackburnand to share it21:26
@sonney2kwell gsomix could do now :D21:28
-!- gsomix [~gsomix@] has quit [Quit: Ex-Chat]21:34
-!- gsomix [~gsomix@] has joined #shogun21:34
puffin444Hey blackburn21:38
puffin444So currently I have a pull request for ARD kernels21:40
blackburnsonney2k: have you seen I added some very basic structure?21:40
blackburnto tutorial21:40
puffin444I have just added ARD into the parameter selection framework. It's done, but I need to make sure the code is commented and stuff.21:40
puffin444My question is that should I just close the current pull request and add a new one today/tomorrow with the new ARD learning attached?21:41
blackburnpuffin444: I don't mind either way :)21:42
blackburnpuffin444: if this one doesn't break compilation I am ok to merge21:43
puffin444The one up there now will not break compilation.21:44
blackburnwhat is ARD btw?21:44
puffin444I frankly don't know what it stands for :), but it essentially gives to a weight to each feature. That we can focus on the features that matter.21:45
puffin444Automatic Relevance Determination21:46
blackburnhmm I work on something similar right now21:46
blackburnfor multitask :)21:46
blackburnI am currently working on modelselection stuff too btw21:50
@sonney2kblackburn, what do you *not* work on?21:51
-!- n4nd0 [] has quit [Quit: leaving]21:51
blackburnsonney2k: I do not work on nuclear bomb21:51
@sonney2kat least something positive :)21:51
blackburnonly neutron one21:52
puffin444Hey blackburn, would you mind merging the current pr?21:52
blackburnI always say yes :D21:52
CIA-18shogun: puffin444 master * r3385475 / (5 files in 2 dirs): Cleaned up some ARD code. -
CIA-18shogun: puffin444 master * r8246e94 / (8 files in 2 dirs): Added Gradient checking in the GradientModelSelection model. Added -
CIA-18shogun: Sergey Lisitsyn master * reb98b84 / (8 files in 2 dirs): Merge pull request #644 from puffin444/master -
blackburnpuffin444: I have to modify some code realted to you probably21:54
puffin444really? where?21:54
blackburnpuffin444: I am adding modelselectionoutput thing in all modelselection things21:55
blackburnargh may be some set_output would be better21:55
blackburnpuffin444: I am adding modelselectionoutput class21:55
blackburnthat outputs learned model during the process21:56
blackburnI need it to analyze sparsity of learned Ws21:56
-!- alexlovesdata [] has left #shogun []22:12
shogun-buildbotbuild #146 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: puffin444 <>22:34
CIA-18shogun: Sergey Lisitsyn master * rfc2e96c / (13 files in 3 dirs): Introduced ModelSelectionOutput class -
-!- heiko [] has quit [Quit: Leaving.]22:58
-!- puffin444 [62e3926e@gateway/web/freenode/ip.] has quit [Ping timeout: 245 seconds]23:12
shogun-buildbotbuild #147 of deb3 - modular_interfaces is complete: Success [build successful]  Build details are at
--- Log closed Sun Jul 15 00:00:17 2012