Open in new window / Try shogun cloud
--- Log opened Mon Aug 22 00:00:12 2011
-!- f-x [~user@] has joined #shogun00:34
-!- serialhex [] has quit [Remote host closed the connection]00:56
-!- serialhex [] has joined #shogun00:58
blackburnserialhex: no more floating faces :(01:09
CIA-87shogun: Sergey Lisitsyn master * r6e02faf / src/shogun/preprocessor/KernelLocallyLinearEmbedding.cpp : Fixes for kLLE -
serialhexwhy no more floating faces???01:10
blackburnserialhex: they died01:11
serialhexdied??? what happened blackburn???01:12
blackburnok joke :D01:12
serialhexdid you kil the floating faces??? :P01:12
blackburnI'm now trying to the get intrinsic manifold of promoter data01:12
serialhexok, cool01:12
blackburnfancy-mancy-pancy bioinformatics hehe01:13
blackburnserialhex: how are you?01:14
serialhexgood good, and yourself blackburn??01:14
blackburnfine, soon starting to studying again01:15
serialhex...and wondering what happened to my local branch of shogun... :(01:15
blackburnwhat's up with your branch?01:15
serialhexyeah?  classes start on tues for me, what are you taking this simester??01:15
blackburnI don't know :D01:16
serialhexit's saying there are a *BUNCH* of files that i edited & it's all stuff like: shogun/ui/GUIHMM.h and shogun/base/init.h  and i have no reason to edit those files...01:16
blackburnserialhex: hmm try rebase01:17
serialhexyou dont know!?!?!  you're gonna be studying but you dont know what!?!?01:17
blackburnI have fixed warnings in these files01:17
blackburnyeah I will study what I have to study01:17
blackburnno choice is given heh01:17
serialhexniiice :P01:17
blackburnI know I have to study experimental design techniques or so01:18
serialhexit's weird, it says i have those files staged for commit... but i havnt done that :(01:18
serialhexcool cool01:18
blackburnjava ee but my teacher will free me for that01:18
blackburncause I will be working as java ee developer starting from september, the 5th01:19
blackburnwell it would be strange to study java ee again :D01:19
serialhexreally? thats kind of cool!!01:19
blackburnyeah I will have to work about 20-30 per week01:20
blackburnI hope I will have much time for studies, vodka and shogun01:20
blackburnnot much, bad word01:20
serialhexyeah, sufficient works better :D01:22
blackburnand I have to study some optical informatics or so01:22
blackburnsome crazy shit I don't want to do01:23
blackburnand mathematical physics again01:23
serialhexlovely :P01:24
serialhexholy crap dude... the shogun repository is ~23MB!!!!01:26
blackburnserialhex: ^ some promoter embedding via kernel LLE I implemented just now01:27
serialhexooh, perdy dots!! :P01:27
blackburndifferent colors are standing for different organisms01:27
blackburnI don't know if it is good or not but at least something01:28
blackburnwith kPCA I've got gaussian cloud01:28
blackburnnothing to deal with01:28
serialhexso is it like showing diffs in DNA or something?01:28
serialhexhmm, ok01:28
blackburnit uses kernel01:28
blackburnI used weighted degree kernel with degree of 10001:29
blackburnand 150 instances of each organism01:29
blackburnso it should be something like underlying manifold of this data01:29
blackburninput data is some ACTCGACTTTTCACGAC....01:30
serialhexhmm... ok01:30
blackburn150*9 strings of length 60001:30
blackburnI don't really understand anything hahah01:30
serialhexooh, niiice.... i bet that took a while to compute :P01:30
blackburnthe most expensive here was to load data01:30
serialhexoh, i thought it was 150**9, not *9 :P01:30
blackburnah nooo01:31
serialhexyeah... that'd be insane!01:31
blackburnI bet it would never do a half of that embedding01:31
blackburnhmm the most strange thing for me is that according to the embedding all of the promoters are the same01:32
blackburnI should add some crap here and see if it will be farther or so01:32
blackburnserialhex: time for sleep, see you :)01:47
serialhexok, nite blackburn01:48
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]01:48
-!- f-x [~user@] has quit [Remote host closed the connection]01:55
@sonney2kserialhex, problem is I cannot use CONFIG['sitelibdir']06:10
@sonney2kserialhex, the prefix that we might want to install to might be different from /usr/local ...06:10
@sonney2kso I need to set it someow06:11
@sonney2kserialhex, so ruby -e 'require "mkmf"; CONFIG["sitedir"]="/my/prefix"; puts Config::expand("$(sitelibdir)")' should say sth lilike /my/prefix ...06:56
-!- f-x [~user@] has joined #shogun08:02
-!- f-x [~user@] has quit [Remote host closed the connection]08:04
CIA-87shogun: Soeren Sonnenburg master * r676fcae / (src/Makefile.template src/configure): attempt to determine ruby extensions dir and add make install target -
CIA-87shogun: Soeren Sonnenburg master * rddac142 / (src/Makefile.template src/configure): add option --rubydir to override ruby installation path -
CIA-87shogun: Soeren Sonnenburg master * r0964f03 / src/shogun/base/Parallel.h : it should be NULL in the mutex case -
CIA-87shogun: Soeren Sonnenburg master * rdfba7da / src/shogun/base/Parallel.h : fix mutex macro -
-!- blackburn [~blackburn@] has joined #shogun10:47
blackburnsonney2k: eeh?!10:50
blackburnuh :(10:52
blackburndid you fix it?10:53
blackburnparallel issue?10:54
@sonney2kblackburn, no11:03
@sonney2kblackburn, it does neither work with mutex nor with spinlocks11:04
blackburnI don't know where regression happen11:04
blackburnare you sure it is related to locks?11:04
@sonney2k#include <shogun/base/init.h>11:04
@sonney2k#include <shogun/features/Labels.h>11:04
@sonney2kusing namespace shogun;11:04
@sonney2kint main(int argc, char** argv)11:04
@sonney2kCLabels* l = new CLabels(10);11:04
@sonney2kreturn 0;11:04
@sonney2kthis crashes11:05
@sonney2kwell it crashes in the call to CSGObject()11:05
blackburnvalgrind shows strange error11:05
blackburnokay I'll check11:05
blackburnmodular works..11:05
@sonney2kblackburn, no11:05
@sonney2kcompile yes but no more11:05
blackburnpython examples works here11:06
@sonney2kblackburn, I guess you should git bisect11:06
blackburnI'll disable ref counting11:07
blackburnand check11:07
@sonney2kthough I suspect that it is working before the spinlock macro change11:07
@sonney2kblackburn, good idea11:07
blackburnuh a lot of warnings from DynamicObjectArray11:09
blackburnwhen compiling w/o ref counting11:09
@sonney2kand does it work?11:13
blackburnsonney2k: no, disabling ref counting didn't help11:58
blackburnsonney2k: do you have any idea what caused this?12:16
blackburnmaking git bisect now12:19
blackburnokay sgobject things caused this12:58
blackburnbut why12:58
@sonney2kblackburn, which commit is it?13:03
blackburnthe problem is that revision is not compiling13:04
blackburnI fixed it in the next commit13:04
blackburnso probably better that13:05
@sonney2kblackburn, but that commit is unrelated13:06
@sonney2kblackburn, so ed1846562fef047cca1847d16d79dd3ee8092ac2 is the problem?13:06
@sonney2kblackburn, why did you change it to not using a ptr?13:08
blackburnchanged what?13:08
@sonney2kI mean the &lock instead of lock in the macro?13:08
blackburncause parameter should be ptr13:09
blackburnbut in sgobject it is instance13:09
@sonney2kI mean that is not the problem - it is just confusing when the function before was taking ptr's13:09
@sonney2kthat is why I would have kept the ptr13:10
@sonney2kthen one woul PTHREAD_UNLOCK(&m_ref_lock);13:10
blackburnyeah may be13:10
blackburnbut it is not a problem13:10
@sonney2kI don't see the problem...13:14
blackburni've reverted the commit13:14
blackburnwill try to run13:14
@sonney2kblackburn, the other option is to comment parts of the code until it works13:17
blackburnworks now13:17
blackburnsonney2k: I think it is better to push revert and then commit it then it works13:17
blackburnit's kinda bad that shogun is not working now13:18
blackburndo you agree?13:18
CIA-87shogun: Sergey Lisitsyn master * r0020af8 / (src/shogun/base/SGObject.cpp src/shogun/base/SGObject.h):13:19
CIA-87shogun: Revert "SGObject mutex to generic lock transition"13:19
CIA-87shogun: This reverts commit ed1846562fef047cca1847d16d79dd3ee8092ac2. -
blackburnsonney2k: interesting thing: libshogun examples can't run cause they don't know anything about PTHREAD_LOCK_T..14:34
blackburnI have no idea how to avoid this15:00
-!- in3xes [~in3xes@] has joined #shogun15:07
-!- f-x [~user@] has joined #shogun16:40
-!- in3xes_ [~in3xes@] has joined #shogun17:13
-!- in3xes [~in3xes@] has quit [Ping timeout: 258 seconds]17:16
-!- in3xes_ is now known as in3xes17:44
CIA-87shogun: Sergey Lisitsyn master * r904b1a8 / examples/undocumented/python_modular/ : Added kLLE example -
-!- alesis-novik [~alesis@] has quit [Quit: Leaving]18:48
-!- in3xes_ [~in3xes@] has joined #shogun18:48
-!- in3xes [~in3xes@] has quit [Ping timeout: 240 seconds]18:52
-!- f-x [~user@] has quit [Remote host closed the connection]19:00
CIA-87shogun: Sergey Lisitsyn master * r876909b / src/shogun/preprocessor/DimensionReductionPreprocessor.h : Added AUTO_TARGET_DIM option for dimensionality reduction preprocessor -
-!- serialhex [] has quit [Remote host closed the connection]19:19
-!- heiko [] has joined #shogun19:21
-!- heiko [] has quit [Ping timeout: 260 seconds]19:25
-!- in3xes_ is now known as in3xes19:36
blackburnsonney2k: should -larpack or so be in params of of libshogun?20:15
blackburnfor arpack test being compiled20:15
@sonney2kblackburn, why should examples know anything about PTHREAD_LOCK_T?20:22
@sonney2kblackburn, no we need to modify the code then to don't have library calls in the .h file20:23
@sonney2kblackburn, then this can be avoided20:23
blackburnsonney2k: arpack.h have no library calls20:24
blackburnbut not visible for some reason20:24
blackburncan't understand why20:24
@sonney2kthen I don't understand what you mean20:24
blackburnI tried to make mathematics_arpack.cpp example20:24
blackburnbut arpack_dsaeupd_wrap routine 'was not declared'20:25
@sonney2kblackburn, could it be that you don't include lib/config.h in the file where you define arpack?20:29
blackburnhmm I didn't20:31
blackburndidn't help :(20:32
blackburnsonney2k: any other ideas?20:35
blackburnoh I see20:36
blackburnwrong order of includes :)20:37
@sonney2kblackburn, err no20:37
@sonney2kthen some include is missing somewhere20:37
blackburnsonney2k: I placed include of config *after* ifdefs20:38
@sonney2kbut lib/config.h must be included first that is right20:38
@sonney2kblackburn, what about the PTHREAD_LOCK_T thing?20:38
@sonney2kblackburn, btw is that perhaps the reason for the crash we were seeing?20:39
* sonney2k tries20:39
@sonney2kunfortunately not20:40
blackburnsonney2k: I can't fix it20:41
blackburnI've reverted sgobject transition20:42
blackburnnow it uses mutex20:42
blackburnwhen I try to use PTHREAD_LOCK_T there - examples fails20:42
blackburnI don't know why20:42
CIA-87shogun: Sergey Lisitsyn master * r80149f4 / (7 files in 3 dirs): Improved and renamed arpack wrapper, added libshogun arpack example -
@sonney2kblackburn, I found the problem20:53
@sonney2kbug in configure script20:53
blackburnto which issue is it related?20:54
@sonney2kblackburn, and only possible because we are doing library calls in .h file20:54
@sonney2kto the lib/config.h include one and the library calls in .h file one20:54
@sonney2kI will commit a revert reverted SGObject version with the & replaced soon20:55
blackburnsonney2k: so should config.h be included in prior of every HAVE_*?20:59
@sonney2kblackburn, yes20:59
@sonney2konce per file at most :)21:00
blackburnyeah I see..21:00
@sonney2kblackburn, of course we could avoid this if we had no ifdef HAVE_XXX in a .h file21:03
@sonney2kbut that is not possible21:03
blackburnsonney2k: so did you manage to make generic lock work in sgobject?21:04
@sonney2kblackburn, configure was not writing out the HAVE_PTHREAD to lib/config.h21:06
@sonney2kbut only setting it in DEFINES (arguments passed to g++)21:07
@sonney2ksame for SPINLOCKS btw21:07
blackburnmy bug :D21:07
blackburnhmm but spinlock is yours21:07
@sonney2kI copy pasted from PTHREAD :)21:07
blackburnsonney2k: your email is wonderful thing to start a day with haha :D21:08
blackburnbtw I had a dream I killed someone and got email What have you done :D21:09
@sonney2kblackburn, I spend all my spare time on hunting this bug today too - so you are not alone...21:09
blackburnsonney2k: how soon will you receive new computer?21:10
@sonney2kblackburn, no idea - hopefully this week21:10
CIA-87shogun: Sergey Lisitsyn master * ra030b01 / (9 files): Added necessary include of config header to preprocessors -
blackburnsonney2k: I've got this picture for the promoter data:
blackburnhave you any ideas what axes could mean? :D21:12
@sonney2kblackburn, for one I would expect GC frequency21:20
blackburnsonney2k: hm! I should check, thanks21:21
blackburnsonney2k: what do you think about storing consts this way?
blackburnit is used in java commonly21:33
-!- heiko [] has joined #shogun21:34
blackburnfor examples I can do something like21:34
blackburnltsa = LocalTangentSpaceAlignment()21:34
blackburnwithout any import except ltsa itself21:34
blackburnfor example*21:34
@sonney2kblackburn, shouldn't it be static const int32_t AUTO_TARGET_DIM ?21:34
blackburnwhat is the difference?21:35
-!- heiko [] has quit [Client Quit]21:35
blackburnhmm yes21:36
CIA-87shogun: Soeren Sonnenburg master * r728d1de / (3 files in 2 dirs):21:36
CIA-87shogun: Merge pull request #307 from alesis/gmm21:36
CIA-87shogun: Bug fix for GMM. -
-!- heiko [] has joined #shogun21:36
CIA-87shogun: Soeren Sonnenburg master * ra715fc7 / (5 files in 3 dirs):21:36
CIA-87shogun: Merge pull request #308 from frx/streaming_vw21:36
CIA-87shogun: VW application and option to save predictions to a file -
CIA-87shogun: Soeren Sonnenburg master * re7aed82 / src/interfaces/ruby_modular/modshogun.rb : don't require pry -
CIA-87shogun: Soeren Sonnenburg master * r4d7f40a / (src/shogun/base/SGObject.cpp src/shogun/base/SGObject.h): use generic defines for pthread spinlock / mutexes -
CIA-87shogun: Soeren Sonnenburg master * ra57de60 / (4 files in 3 dirs): fix the mutex/spinlock bugs -
CIA-87shogun: Soeren Sonnenburg master * rd32fded / (src/shogun/base/SGObject.cpp src/shogun/base/SGObject.h):21:37
CIA-87shogun: move library calls in SGObject into .cpp files and un-inline respective21:37
CIA-87shogun: functions -
CIA-87shogun: Sergey Lisitsyn master * r706bca0 / src/shogun/preprocessor/DimensionReductionPreprocessor.h : int32_t const -> const int32_t in dimreductionpreprocessor -
blackburnsonney2k: so what are the tasks we have to finish?21:41
blackburnthere should be <50 warning now, seems various examples is what we have to do21:42
@sonney2kblackburn, get the test suite to work fix remaining warnings, fix / port examples for all new languages, get installer to work for new languages, write  / update make release script21:47
@sonney2kI am working on the install scripts currently21:49
@sonney2kblackburn, would be good if you could get the warnings down to 021:49
blackburnhmm okay21:49
@sonney2k(and you don't need a buildbot for that - just compile python_modular and you see all the many)21:49
blackburnsure but I usually disable doxygen hah21:49
@sonney2kI never do21:50
@sonney2kI guess I like to suffer21:50
@sonney2kblackburn, btw it doesn't even compile here w/ doxygen on21:51
blackburnehh why?21:51
@sonney2kblackburn, some unicode errror you\u22c5' ?21:51
@sonney2kno idea what that could be21:51
@sonney2kno idea21:51
blackburnokay will try to detec21:51
@sonney2kblackburn, found it21:52
@sonney2kwas ?21:52
@sonney2kand in your commit21:52
CIA-87shogun: Soeren Sonnenburg master * r1a48148 / src/shogun/preprocessor/MultidimensionalScaling.h : fix unicode error -
blackburnno idea what it is21:56
-!- heiko [] has quit [Quit: Leaving.]21:59
@sonney2kblackburn, it looks like a broken pixel on my display21:59
-!- heiko [] has joined #shogun21:59
@sonney2kI was first trying to remove the dust from my display but it is an actual unicode character pixel ?22:00
@sonney2kheiko, how is it going?22:03
heikosonney2k, hi, the stuff in running, no more errors22:03
heikobut it leaks22:03
@sonney2kthe first sentence is exactly what I wanted to hear :)22:04
heikothere are some minor memory leaks which i hope to fix soon :)22:04
heikoThough its kind of complicated to add a migration now, but you will see. Let me just hunt down the memory leaks .)22:04
@sonney2kheiko, I guess you will be just in time for when the new buildbot arrives22:05
heikowhen will that be? :)22:06
blackburnsonney2k: have you seen doc in vojtech's toolkit?22:08
@sonney2kno - did you?22:08
blackburnI want similar22:08
@sonney2kheiko, somewhen this week22:08
@sonney2kblackburn, does it contain formulas?22:09
blackburnit contains short description22:09
blackburnand it is in latex -> pdf22:09
@sonney2kblackburn, yeah but then jsut write formulas22:09
@sonney2kI did...22:09
blackburnno, I want to choke seeing doxygen formulas22:09
blackburndon't like it22:11
@sonney2kblackburn, if you want to spend all your life keeping the documentation up-to-date - feel free22:11
@sonney2kblackburn, or write a book even22:11
blackburnwhy? do methods change?22:11
@sonney2kall the time22:12
@sonney2kand no one updates documentation22:12
blackburne.g. LLE stays LLE22:12
@sonney2kit is already tough enough to update the doc inside the source code22:12
blackburnno chance it will be changed22:12
@sonney2kblackburn, haha - like PCACut you mean?22:12
@sonney2kor Classifier22:13
blackburnnot the API doc22:13
@sonney2kblackburn, the method itself?22:13
@sonney2kfor that a pointer to a paper is all that is needed22:13
blackburneverytime I see vojtech's toolbox description I cry and want the same for shogun :(22:14
@sonney2kblackburn, no one wants to write documentation - so if you don't do it it will always be like this22:15
blackburnpossibly it is because I'm a latex fan22:15
-!- in3xes_ [~in3xes@] has joined #shogun22:16
blackburnsonney2k: btw I have a nice doc for my preprocessors :P even without mistakes haha22:17
@sonney2kserialhex proof-read it?22:17
@sonney2kheiko, this week I hope22:17
heikosonney2k, ok22:17
@sonney2kheiko, my machine is just too unstable ...22:17
@sonney2kso I bought a new one22:18
heikook, cool that we got a new one then22:18
heikowill you install it at home?22:18
@sonney2kheiko, now with 16G of memory - btw whose price dropped to 52 EUR now22:18
heikooh no, i paid too much :)22:18
heikobut my 4gb are cool now, everything much faster22:18
blackburnsonney2k: do you know everything fails now?22:19
-!- in3xes [~in3xes@] has quit [Ping timeout: 240 seconds]22:19
-!- heiko [] has quit [Quit: Leaving.]22:21
@sonney2kblackburn, life is a total failure - in the end we all fail (aka die) or what are you talking about?22:21
blackburnthey all fail now22:22
-!- heiko [] has joined #shogun22:22
@sonney2kblackburn, no they work here22:23
blackburnI'm up-to-date22:24
@sonney2kme too22:24
blackburnall examples fails with segfaults22:24
@sonney2kblackburn, all work through nicely22:24
@sonney2klibshogun, python_modular...22:24
blackburnok will try to rebuild22:25
blackburnand reconfigure22:25
-!- heiko [] has left #shogun []22:30
blackburnsonney2k: still fails22:33
blackburn==2650== Invalid write of size 122:34
blackburn==2650==    at 0x43FF689: shogun::CGaussianKernel::set_compact_enabled(bool) (GaussianKernel.h:118)22:34
@sonney2kblackburn, are you at 1a481483e709e3d1a0010e7c8d5bf8bfdeb4333c22:34
@sonney2kgit clean -dfx22:34
@sonney2k./configure --interfaces=python_modular   +otheropts22:35
@sonney2kmake install22:35
blackburnI already did distclean and configure and make22:35
blackburntrying again22:35
-!- heiko [] has joined #shogun22:38
@sonney2kblackburn, I also rebuild /reinstalled everything - works all nicely22:41
blackburnsonney2k: Segmentation fault22:43
blackburnsonney2k: okay got it22:44
blackburnI didn't recompile examples22:44
blackburnall ok now22:44
@sonney2kblackburn, next time just type make check-examples22:47
@sonney2kthat will do it for you22:47
blackburnyeah it is what I did22:47
@sonney2kblackburn, so now you can finally fix examples :)22:52
@sonney2kerr warnings22:52
blackburnyeah will start in a min22:52
blackburnnot really much left22:52
@sonney2kI have lots of in preprocessors...22:52
@sonney2k10-20 already22:52
blackburnyeah it is because of kpca refactoring22:53
-!- heiko [] has quit [Read error: Connection reset by peer]23:08
-!- heiko [] has joined #shogun23:09
heikosonney2k, my example is now memclean :) :)23:24
heikonow for some more complex tests23:30
blackburnwarnings -> 023:42
blackburnverrrrry soon hah23:42
@sonney2kheiko, great!23:46
@sonney2kgo guys go!23:46
blackburn10 to go23:47
heikosonney2k, I will open a pull request so that you can read a bit what I did.23:47
heikoI will polish it a bit more, don't yet merge23:48
@sonney2kheiko, ok23:48
@sonney2kblackburn, btw regarding documentation - I had a look at what you all wrote. I think it really needs some improvement in the header: like formula of the problem that is to be solved and when it is most useful to use this method plus a pointer to a paper.23:49
blackburnsonney2k: are you kidding? we will never finish if I writing true doc23:49
@sonney2kblackburn, what is a true doc?23:50
blackburnwith formulas, etc23:50
blackburnI will improve doc but writing all of it now - it is impossible23:50
blackburnsonney2k: or what do you mean?23:52
blackburnthe whole shogun or my preprocs?23:52
blackburnif the whole shogun ^23:52
@sonney2kI didn't say that you have to do it *now* - I said that documentation is worse than average - which is e.g.
blackburnyeah sure we have worst doc ever23:53
@sonney2knot true23:55
@sonney2kwe don't have a book or tutorial23:55
@sonney2kit is more like a reference for someone who knows about ML23:55
blackburn2 warnings left23:56
@sonney2kblackburn, writing a tutorial style book for the methods inside of shogun is like 1000 pages23:58
@sonney2kgood luck when attempting that23:59
blackburncalm down I don't want to write it23:59
@sonney2kblackburn, you wine all the time23:59
--- Log closed Tue Aug 23 00:00:15 2011