Open in new window / Try shogun cloud
--- Log opened Fri Aug 19 00:00:00 2011
@sonney2kyeah epic failure powered by w. gorbachew00:00
blackburnI hope we are only two subscribers of trac00:01
blackburnI've generated a lot of mails00:01
blackburnhey he is m. :)00:01
@sonney2kbut w - wodka00:02
@sonney2kor just turn it upside down00:02
@sonney2klook at this everything fails but test still claims everything is good00:02
blackburnwodka gorbachew nice00:02
shogun-buildbotbuild #59 of octave_static is complete: Failure [failed test]  Build details are at  blamelist: sonne@debian.org00:03
blackburnTest successful: failure00:03
@sonney2keven better it works locally :D00:04
blackburnsonney2k: wtf is:00:05
blackburnwhy it is in examples00:06
@sonney2kblackburn, what is the problem?00:07
@sonney2kblackburn, when running tests - should it run all tests and then fail at the end if one of them failed?00:07
@sonney2kor should it fail immediately?00:07
-!- blackburn [~blackburn@] has quit [Read error: No route to host]00:09
-!- blackburn [~blackburn@] has joined #shogun00:10
blackburnmake leaded to reboot heh00:10
blackburnsonney2k: what is it, why it is root of examples?00:11
@sonney2kblackburn, when running tests - should it run all tests and then fail at the end if one of them failed?00:11
@sonney2k or should it fail immediately?00:11
@sonney2kI tend to say at the end00:12
blackburnat the end00:12
blackburnif we broke something in the middle00:12
blackburnwe will not see if something is broken too00:12
blackburn-> harder to debuuug00:12
CIA-87shogun: Soeren Sonnenburg master * r5a0fd39 / (9 files in 9 dirs): exit 1 on error at the end of running -
@sonney2kthere we go00:15
@sonney2kahh root of examples00:16
@sonney2kblackburn, bug :)00:16
@sonney2kshould be in python_static00:16
blackburnyeah that's why I was tending to launch nuclear missile00:17
@sonney2kblackburn, btw it would be nice to see the actual error00:17
@sonney2knot just error00:17
@sonney2kat least at the end00:17
@sonney2kblackburn, this output is pretty useless00:17
@sonney2kERROR - > great00:18
blackburnhooray to the error! hah00:18
blackburnno idea how to do that00:18
* sonney2k is a dedicated ERROR collector00:18
@sonney2kblackburn, it is easy - just store output in an array and print it at the end00:19
@sonney2kI will do it but not now (need sleep...)00:19
@sonney2kblackburn, I don't understand why tests are failing on the buildbot00:20
@sonney2kthey all work here locally...00:20
blackburnbtw is python static working now?00:20
blackburnhaven't checked for a while00:20
shogun-buildbotbuild #69 of cmdline_static is complete: Failure [failed test_1]  Build details are at  blamelist: sonne@debian.org00:27
@sonney2kit should00:41
@sonney2kblackburn, ahh it needs the data submodule to be checked out00:49
@sonney2kthat is why things fail00:49
blackburnthought you're sleeping :D00:49
blackburnsonney2k: I'm now having a zoo of 9 dim reduction preprocessors :D00:51
-!- shogun-buildbot [] has quit [Quit: buildmaster reconfigured: bot disconnecting]00:53
-!- shogun-buildbot [] has joined #shogun00:53
@sonney2kblackburn, don't forget to feed all your animals00:54
blackburnwant to start implementing sammon but still have to fix pcas and get some promoter visualization00:54
CIA-87shogun: Soeren Sonnenburg master * r5ed8a9a / examples/undocumented/python_static/ : don't run for loop in a sub shell -
@sonney2kso now really everything should fail :)00:57
@sonney2kblackburn, I have to figure out how to use the github trigger00:57
@sonney2kthen directly after a commit the buildbot will start its job00:57
blackburngithub trigger?00:57
@sonney2khmmhh data is big01:06
@sonney2kthe git clone takes ages...01:06
shogun-buildbotbuild #70 of cmdline_static is complete: Success [build successful]  Build details are at
shogun-buildbotbuild #61 of python_static is complete: Failure [failed test_1]  Build details are at  blamelist: sonne@debian.org01:20
@sonney2kblackburn, alright - now it in principle works
@sonney2kthere is one python test failing01:21
@sonney2kbut we indeed don't support that for python01:21
@sonney2kso removing the example would fix it01:21
blackburnsonney2k: you will be laughing but my mds is the slowest one ever01:22
blackburnI didn't know there is fast tecnhique for double centering01:22
blackburnI did it using blas01:22
blackburnso stupid01:23
@sonney2kanyway I am dead tired now01:23
@sonney2kcu tomorrow!01:23
@sonney2ktoday even01:23
blackburnsee you01:23
blackburnI will be fixing mds01:23
serialhexhaving fun fixing mds blackburn?? :P01:25
blackburnserialhex: yeah and wondering how stupid I am01:25
serialhexhey,you knew there was something broken (or at least not working well)01:26
serialhexand that's the first step to making it better! :D01:26
blackburnI noticed it because pca makes it for 0.02s01:26
blackburnand mds for 2s :D01:27
serialhexthat's crazy!!!01:31
serialhexhey, the python rand() func, when you call it like rand(7) does it give you an int between 0 & 6 blackburn??01:32
blackburnis there any rand() function?01:33
serialhexand what would "data=rand(dim, dim)" do???01:33
serialhexhmm... ok01:33
blackburnhmm may be it is numpy function01:34
blackburnlet me check01:34
blackburnserialhex: yeah numpy func01:34
serialhexahh, it is numpy!01:34
serialhexyes :P01:34
blackburnreturns dim*dim matrix of uniform distributed randoms01:34
serialhexhmm, ok01:34
blackburnrandint for your task01:34
blackburnnot numpy01:34
blackburnor so01:34
serialhexwell i'm converting classifier_custom_kernel_modular so i ned to know how those funcs work to translate them properly01:35
blackburnso it is matrix of dim*dim [0,1] uniform distr blabla01:35
serialhexok cool01:36
serialhexi think i know how to solve this stuff!  :)01:37
blackburnreally funny01:37
blackburnI thought MDS is pretty slow01:37
blackburnI made distance calculation parallel01:38
blackburnI made landmark approximation01:38
blackburnand it is fucking awful I did 2 matrix-matrix multiplications01:38
blackburnwhen no need to do it01:38
serialhexHA!!! lol01:39
blackburnserialhex: another one fact about my stupidity01:41
blackburnwe have center_matrix function in math01:41
blackburnand I didn't use it oooooh01:42
serialhexHAHA!!!  :)01:42
serialhexit's ok, you'll live! :P01:43
blackburnwhat a shame01:43
serialhexwell i just had to look up the interface for the random number generator that *I* wrote oh-so-many months ago :P01:44
blackburnhey isn't there any lib for that?!01:44
serialhexprobably, but we have one in shogun that i wrote at the beginning of summer, remember??01:46
blackburnnot really01:47
blackburncan't remember01:47
serialhexreally?  you made the graph for me b/c i couldn't figure out how TF to do it! :P01:49
serialhexyes... it was a normal random function... like 10k iterations to see if it was truely 'normal'01:50
blackburnyes yes01:50
serialhex(or normal enough)01:50
serialhexyeah, seems you forgot too :P01:51
serialhexgeez!  i've had like 3 cups of coffe and a cup of tea and i'm kinda tired!!! :(01:54
-!- in3xes [~in3xes@] has quit [Remote host closed the connection]02:11
blackburndid it02:34
serialhexhows it run now blackburn??02:36
blackburn20-30 times faster02:36
CIA-87shogun: Sergey Lisitsyn master * r46db854 / src/shogun/preprocessor/MultidimensionalScaling.cpp : Efficient matrix centering instead of crappy dgemms for MDS -
serialhexthats sweet!02:38
blackburnserialhex: 15 times faster for 100002:39
blackburnand 3x less memory usage..02:39
blackburnokay I have to sleep now02:40
blackburn4-40 here02:40
serialhexDAMN!!!!! :P02:40
serialhexok man, g'nite and sleep well!!!02:40
blackburnsee you02:40
blackburngood evening? :)02:40
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]02:40
CIA-87shogun: Soeren Sonnenburg master * r90c20fe / examples/undocumented/python_static/ : remove unsupported example -
shogun-buildbotbuild #63 of python_static is complete: Success [build successful]  Build details are at
CIA-87shogun: Soeren Sonnenburg master * r9475428 / (9 files in 9 dirs): collect all errors in error.log and show them at the end in detail -
shogun-buildbotbuild #73 of cmdline_static is complete: Failure [failed test_1]  Build details are at  blamelist: sonne@debian.org10:07
-!- blackburn [~blackburn@] has joined #shogun10:12
blackburnsonney2k: have you read about my shame? :D10:14
shogun-buildbotbuild #64 of python_static is complete: Failure [failed test_1]  Build details are at  blamelist: sonne@debian.org10:16
CIA-87shogun: Soeren Sonnenburg master * rdd04e8d / (9 files in 9 dirs): exit 0 upon examples running all OK -
CIA-87shogun: Soeren Sonnenburg master * r675a2a7 / src/Makefile.template : fix r static doc install path -
@sonney2kblackburn, heroic work yes10:19
@sonney2kblackburn, however I think I can beat you in all aspects of dumbness10:20
@sonney2kbut not today10:20
@sonney2ktoday I am brilliant :D10:20
blackburnI should profile my algos asap10:22
blackburnsonney2k: the most funny moment is that eigenproblem takes 0.7s for 10000 hah10:25
blackburnso the bottleneck was NOT the eigenproblem10:25
shogun-buildbotbuild #74 of cmdline_static is complete: Success [build successful]  Build details are at
blackburnsonney2k: do you know how to do (I-W)(I-W)' without dgemm?10:28
shogun-buildbotbuild #65 of python_static is complete: Success [build successful]  Build details are at
blackburnI'm now getting unsure there is no fast way to do such ops10:28
shogun-buildbotbuild #65 of octave_static is complete: Success [build successful]  Build details are at
CIA-87shogun: Soeren Sonnenburg master * r201a958 / (12 files in 11 dirs):10:47
CIA-87shogun: add script to libshogun to follow the same testing principle10:47
CIA-87shogun: and really exit 1 on failure -
shogun-buildbotbuild #80 of libshogun is complete: Failure [failed test_1]  Build details are at  blamelist: sonne@debian.org10:51
CIA-87shogun: Soeren Sonnenburg master * r316fcc9 / examples/undocumented/libshogun/ : add escaped quotes to libshogun check -
-!- heiko [~heiko@] has joined #shogun11:31
heikoshogun-buildbot, status all11:45
shogun-buildbotcmdline_static: idle, last build 56m49s ago: build successful11:45
shogun-buildbotcsharp_modular: idle, last build 13h13m49s ago:11:45
shogun-buildbotjava_modular: idle, last build 13h10m48s ago:11:45
shogun-buildbotlibshogun: idle, last build 29m26s ago: failed test_111:45
shogun-buildbotlua_modular: idle, last build 13h13m49s ago:11:45
shogun-buildbotnightly_all: idle, last build 13h53m51s ago: build successful11:45
shogun-buildbotnightly_none: idle, last build 13h53m51s ago: build successful11:45
shogun-buildbotoctave_modular: idle, last build 12h55m48s ago: failed test test_111:45
shogun-buildbotoctave_static: idle, last build 1h03m04s ago: build successful11:45
shogun-buildbotpython_modular: idle, last build 13h13m49s ago:11:45
shogun-buildbotpython_static: idle, last build 57m27s ago: build successful11:45
shogun-buildbotr_modular: idle, last build 13h13m49s ago:11:45
shogun-buildbotr_static: idle, last build 1h11m15s ago: failed test11:45
shogun-buildbotruby_modular: idle, last build 13h13m49s ago:11:45
heikosonney2k, around?11:45
@sonney2kheiko, yes11:46
heikosonney2k, did the patch work?11:46
heikosonney2k, cool 8-)11:46
heikosonney2k, however there are still problems:11:46
@sonney2kI am now trying hard to get the buildbot to work with all the make check-examples stuff11:46
@sonney2k(for the last 20rhs already...)11:46
heikothe hdf5 file leaks memory, even without serialization going on, json also, xml does read from uninit memory11:47
heikosonney2k, oh no, were you awake the whole night?11:47
@sonney2kheiko, no11:48
heikopuh :)11:48
@sonney2kheiko, yeah I've seen your comment11:48
@sonney2kwe need migrations more urgently now11:48
heikoI will go for it11:48
@sonney2kso ignore for the time being11:48
CIA-87shogun: Soeren Sonnenburg master * r4d140a3 / (6 files): fix and add return 0 to examples missing the statement -
blackburnsonney2k: how can I set parallel for CDistance?12:13
@sonney2k(like for ever sgobject)12:14
blackburnit is public!12:14
CIA-87shogun: Soeren Sonnenburg master * ra4e2ac1 / examples/undocumented/libshogun/ : forgot escaping args -
shogun-buildbotbuild #83 of libshogun is complete: Success [build successful]  Build details are at
heikofinally :)12:33
shogun-buildbotbuild #67 of octave_static is complete: Failure [failed test_1]  Build details are at  blamelist: sonne@debian.org12:35
@sonney2khmm Inconsistency detected by dl-close.c: 736: _dl_close: Assertion `map->l_init_called' failed!12:37
heikoehm ?? never saw this12:41
@sonney2kheiko, even better doesn't happen on my notebook but only on the buildbot12:44
heikosonney2k, I love these kind of errors12:44
heikodifferent g++ version?=12:44
@sonney2kheiko, ahh yeah there it is
@sonney2kvalgrind doesn't say anything12:47
* sonney2k is rebuilding octave3.212:50
@sonney2kdoesn't even build :D13:02
@sonney2kand this issue is already filed as bug #63778713:03
blackburnhuh LTSA weight calculation step is slower when parallel13:13
blackburnhey have anybody real multiprocessor machine? heiko? sonney2k?13:28
heikoblackburn, i have a centrino-duo13:29
heikoi think there are two cores13:29
blackburnI see13:29
heikobut not two processors13:29
blackburnsame here13:29
CIA-87shogun: Soeren Sonnenburg master * r936bcdf / src/shogun/structure/DynProg.h : fix a few warnings in DynProg -
blackburnfuck yeah13:36
blackburnspinlock rocks13:37
shogun-buildbotbuild #72 of r_static is complete: Success [build successful]  Build details are at
shogun-buildbotbuild #68 of octave_static is complete: Success [build successful]  Build details are at
CIA-87shogun: Sergey Lisitsyn master * rc0f76ee / (2 files): Made neighbor search parallel in LocallyLinearEmbedding -
CIA-87shogun: Sergey Lisitsyn master * r92c21ec / (4 files): Spinlock instead of mutex -
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]14:09
shogun-buildbotbuild #70 of octave_modular is complete: Success [build successful]  Build details are at
-!- blackburn [~blackburn@] has joined #shogun17:48
CIA-87shogun: Sergey Lisitsyn master * r77c1f2b / (3 files in 3 dirs): Made 100 warnings left -
blackburnoh it seems I got wrong17:53
blackburnheiko: ich heisse sergey18:42
heikoblackburn, learning german for mariage? :D18:42
blackburnin fact I really want to learn german18:43
blackburnbut english is prior of course haha18:43
heikowhat for?18:45
heikoi mean german18:45
blackburnidk may be possible country to live :)18:45
blackburnor just for interest18:45
blackburnit is pretty bad here you know :D18:45
-!- heiko [~heiko@] has left #shogun []19:28
@sonney2kblackburn, did you fix these already
blackburnthese ones are new19:47
@sonney2kblackburn, want to get back to 2500 warnings?19:49
blackburndon't understand what is you are talking about19:49
@sonney2kblackburn, perestroika19:49
blackburnNULL -> 0 is not a hard conversion19:49
CIA-87shogun: Sergey Lisitsyn master * r4824398 / (2 files): Fixed spinlock inits -
CIA-87shogun: Soeren Sonnenburg master * rdf7739a / (2 files): fix ppwm example and disable crashing parts in structure_* -
@sonney2kblackburn, alright octave_modular and python_modular are working 100% w/ all examples20:17
@sonney2kr_modular cannot work until swig is fixed20:18
@sonney2kso what remains to do is proper installation for java_modular, lua_modular, csharp_modular and ruby_modular20:18
@sonney2kand then running the examples we have / and their ports20:18
shogun-buildbotbuild #65 of r_modular is complete: Success [build successful]  Build details are at
shogun-buildbotbuild #62 of python_modular is complete: Success [build successful]  Build details are at
blackburnI would like to remove all warns and finish pca/kpca20:28
blackburnbut need actual list of warnings20:29
blackburnsonney2k: should we get applications back to work?20:44
shogun-buildbotbuild #66 of r_modular is complete: Failure [failed test_1]  Build details are at  blamelist: blackburn91@gmail.com20:58
@sonney2kblackburn, yes20:59
@sonney2kbut first the examples20:59
blackburna lot of work still..20:59
@sonney2kblackburn, yes but I am positively surprised that octave/python_modular are good to go already21:00
@sonney2kif we have migrations we could get the old tests for python back to work and then it is even clear that we have no regressions21:00
blackburnI guess we have21:00
@sonney2kI don't really expect any21:02
@sonney2k1-2 maybe21:02
@sonney2kblackburn, can you write an analogous script like I wrote for python_modular for some other $LANG?21:09
@sonney2kruby / java / csharp / lua?21:09
blackburnyeah but I have to finish kpca first :(21:09
blackburnI don't understand what should I do with kpca ..21:12
blackburnsonney2k: why preimage is always computed here in stprtool21:14
blackburnI really lost the idea of kpca..21:14
@sonney2kblackburn, the idea is easy - just do pca in kernel feature space21:15
@sonney2kthe inverse mapping is nice to have but not essential21:16
@sonney2kand I don't know how one could compute it if inputs are not just real valued numbers21:16
blackburnso in vojtechs code kernel is used21:16
@sonney2k(kpca can work with anything)21:16
blackburnwhat is the lhs and rhs for tihs kernel?21:16
@sonney2klike strings21:16
@sonney2klhs == train data, rhs == train or test data21:17
blackburnso I have transformation matrix21:18
blackburnbut what is the kernel in apply? should it be centered?21:18
@sonney2klhs=train & rhs=test data21:21
@sonney2kcentering - i think so21:21
@sonney2klook at vojtechs code he does some magic there21:21
shogun-buildbotbuild #63 of python_modular is complete: Failure [failed test_1]  Build details are at  blamelist: blackburn91@gmail.com21:21
blackburnmodel.Alpha = (eye(num_data,num_data)-J)*A;21:22
blackburnmodel.b = A'*(J'*K*Jt-K*Jt);21:22
@sonney2k^ err that is an old build21:22
blackburnthis things are being computed in init21:22
@sonney2k(I cut my finger badly - so it is currently a bit hard to type)21:23
blackburnare you ok?21:23
@sonney2kyes - no blood on the keyboard yet :)21:23
@sonney2kblackburn, 100 warnings exactly now :)21:26
blackburnI will try to grasp kpca 10-15 min more21:27
blackburnand will remove all of them finally21:27
blackburnno, no centering here21:29
CIA-87shogun: Soeren Sonnenburg master * rd0c6dda / (3 files): Fix documentation in structure. -
CIA-87shogun: Soeren Sonnenburg master * r24610e5 / src/shogun/kernel/SNPStringKernel.h : fix documentation in SNPstringkernel -
CIA-87shogun: Soeren Sonnenburg master * rb6a1b12 / src/shogun/lib/DynInt.h : fix type definitions and properly document types up to 1024bit integers -
CIA-87shogun: Soeren Sonnenburg master * r618179a / src/shogun/base/DynArray.h : remove extern SGIO statement -
CIA-87shogun: Soeren Sonnenburg master * raa7f594 / src/shogun/distributions/HMM.h : document BaumWelchViterbiType -
@sonney2kblackburn, but there must be ... in pca you also center the data before applying pca21:48
blackburnno, kernel which eigenvectors are used is centered21:49
blackburnin apply no centering21:49
shogun-buildbotbuild #7 of nightly_none is complete: Failure [failed test_1]  Build details are at
@sonney2kblackburn, how do we treat this? nightly_none fails (of course) because there are some things required to run the example which are just disabled21:55
@sonney2kblackburn, should I disable running tests for nightly_* ?21:55
@sonney2kand only check if things compile?21:55
blackburnyeah disable it better21:55
shogun-buildbotbuild #7 of nightly_all is complete: Failure [failed compile]  Build details are at
blackburnsonney2k: oh I accidentally did it21:58
@sonney2kdid what21:58
blackburnkpca apply21:58
@sonney2kwhy does nightly_all still fail *grmpf*22:00
@sonney2kI am again too tired to continue just now - I guess I will do java_modular in the morning22:01
blackburnthere is already22:01
@sonney2ka working check.sh22:01
@sonney2kand it needs to integrate with make install too22:05
@sonney2kI guess it is not that hard so when that works c# should be easy too22:06
blackburnwhich should I do?22:06
@sonney2kand then there is only lua and ruby left22:06
@sonney2kblackburn, fix the warnings for now22:06
@sonney2kthat's enough work22:06
blackburnkpca is almost ready22:06
@sonney2kthen I guess we will have errors in java_modular tomorrow22:06
@sonney2kand c#22:06
@sonney2kand we need to complete examples, i.e. convert the ones from python_modular to octave/java/csharp/ruby/lua22:07
@sonney2kat least a number of them22:07
blackburnsonney2k: should cleanup make preprocessor uninitialized?22:08
@sonney2kcleanup should remove everything but essential base stuff22:10
@sonney2kso yes22:10
blackburnyeah truly thing22:11
CIA-87shogun: Sergey Lisitsyn master * r16c77fc / (9 files in 2 dirs): Made PCA preprocessor inherited from DimensionReductionPreprocessor, renamed KernelPCACut to KernelPCA -
CIA-87shogun: Sergey Lisitsyn master * rd88e533 / (2 files): Updated kPCA python modular example -
CIA-87shogun: Sergey Lisitsyn master * r92a2edb / (2 files): Updated kPCA ruby example -
CIA-87shogun: Sergey Lisitsyn master * rae7dc3e / (2 files): Updated kPCA java modular example -
CIA-87shogun: Sergey Lisitsyn master * rf192c44 / (17 files in 8 dirs): Warning fixes -
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]23:40
-!- blackburn [~blackburn@] has joined #shogun23:40
-!- blackburn [~blackburn@] has quit [Client Quit]23:40
--- Log closed Sat Aug 20 00:00:06 2011