Open in new window / Try shogun cloud
--- Log opened Mon Feb 13 00:00:19 2012
-!- axitkhurana [d2d43a6f@gateway/web/freenode/ip.] has quit [Quit: Page closed]00:01
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]00:23
-!- n4nd0 [] has quit [Quit: Leaving]01:23
-!- dfrx [] has joined #shogun05:26
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun09:26
CIA-18shogun: Steve Lianoglou master * r0e2fafe / src/ : (log message trimmed)09:39
CIA-18shogun: 1/2-way fix for r_static install procedure.09:39
CIA-18shogun: * The structure of the package.rds file has been broken for some time,09:39
CIA-18shogun:  this fixes it by adding a $DESCRIPTION['Built'] character vector. It09:39
CIA-18shogun:  seems that this has been required since (I believe, at least) R 2.13.09:39
CIA-18shogun: * Cleaned up code to set RVERSION, PLATFORM, and OSTYPE. The current09:39
CIA-18shogun:  way to build RVERSION is improved and allows for sg to be built against09:39
CIA-18shogun: Soeren Sonnenburg master * rc08a984 / src/ :09:39
CIA-18shogun: Merge pull request #371 from lianos/master09:39
CIA-18shogun: This is a 1/2-way fix for the install procedure for the r_static interface. -
-!- sonne|work [~sonnenbu@] has joined #shogun09:46
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]10:24
-!- blackburn [5bdfb203@gateway/web/freenode/ip.] has joined #shogun10:31
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun10:36
blackburnwiking: im still waiting for JS kernel results :D10:59
wikingblackburn: no fucking way10:59
blackburnit is TOO slow :)10:59
wikingin the meanwhile i've got some new kernels coming up10:59
wikingwe'll see about their results10:59
blackburnreally? which one?10:59
wikingbut yeah a little bit of parallel loop there in the compute function would do some good10:59
wikingsome generalization of js-kernel, like jensen-renyi kernel10:59
blackburnI'm going to merge vlfeat things10:59
wikinginto shogun?10:59
wikinggreat! :>10:59
blackburnyeah I'm thinking about it10:59
blackburnwell maybe just provide some adapter to vlfeat's code10:59
wikingthat'd be great10:59
wikingthe homo kernel map is only one .c code in the vlfeat proj10:59
wikingso could be easily copied i think10:59
wikingand the code is BSD licensed10:59
wikingso no worries there10:59
wikingjust wrap a c++ class around it and that's it i think10:59
blackburnyeah I'm going to do it once I get some vodka10:59
wikingand maybe some stuff you'll need from vl/mathop.h11:00
wiking[master vl] $ wc -l homkermap.c                                                                                                                       [wiking@welitron:~/vlfeat/vl]11:00
wiking     501 homkermap.c11:00
wikingso 501 lines of code+comment11:00
blackburnlooks like it is pretty simple11:00
wikingbut yeah make it nice so that the new JS derivative kernels i could implement fast as well within the homokernelmap11:01
wikingas all the kernels i've checked now is positive definite11:01
wikingand summing kernel11:02
wikingso i'm hoping to be able to produce the homo-kernel-map of it as well11:02
blackburnfor some strange reasone first word of this map makes me smile now11:03
blackburnwould be funny to call it homogaykernelmap11:03
blackburnis there any conceptual difference between JS and JR kernels?11:04
wikingjs is based on shannon entropy11:05
wikingjr is renyi entropy11:05
wikingand renyi entropy is a more generalized version of the classical version of (shannon) entropy in information theory11:05
wikinganyhow i'll run some test before doing anything11:06
wikingi'm just a bit sick now11:06
wikingso everything is slower11:06
-!- blackburn [5bdfb203@gateway/web/freenode/ip.] has quit [Quit: Page closed]11:16
-!- blackburn [5bdfb203@gateway/web/freenode/ip.] has joined #shogun11:16
blackburnwiking: LSVM things looks very promising11:42
blackburnI just checked shogun gsoc idea text about lsvm :)11:44
blackburnsonne|work: damn how to catch you11:45
blackburnwiking: did you already implement a JR kernel?11:48
wikingabout to finish11:48
-!- dfrx [] has quit [Quit: Leaving.]12:32
-!- n4nd0 [] has joined #shogun12:35
wikingblackburn: yeah that'd be cool to do it maybe this year12:35
wikingas gsoc12:35
wikingi mean l-ssvm12:35
blackburnwe should check if alex binder is available for mentoring12:35
wikinghe's not coming around here ?12:40
blackburnno, never seen him here12:40
wikingmailing list i guess then12:40
wikingblackburn: r u doing the homo-kernel-mapping stuff now?12:41
wikingor should i dig myself into it ?12:41
blackburnno, I'm on my job right now12:41
blackburnhah feel free if you want12:41
wikingheheh ok12:41
blackburnwiking: I don't mind you doing anything ;)12:42
wikingeheheheh i was just meaning that let's not do double work if possible :>>12:42
wikingah yeah12:43
wikingcan you accept a pull request?12:43
blackburnwhile I'm working last year undergraduate I have not much time12:43
blackburnyes, sure12:43
wikingok so should i do an official request or can u just pull this one in12:43
blackburn[15:42] <wiking> eheheheh i was just meaning that let's not do double work if possible :>> ---- yeah, we shouldn't :)12:43
wikingit's a little bit of less operations this way12:44
blackburnhmm sure12:44
wikingyou know 2 division vs 1 multiplication ;)12:44
wikingso it should speed up things12:44
wikingbut then again12:44
wikingnot that much imho ;)12:45
blackburnwiking: but I can't it from job, will be able to merge it a little later12:45
blackburndo it*12:45
blackburncan't do it*12:45
wikingno worries12:45
wikingjust don't forget12:45
blackburnjust make a pull request and I won't forget for sure12:45
wikingok so now i'm testing the new kernels :DDD12:45
wikingah ok12:45
wikingso let's see the thingy12:45
wikingi guess it's gonna take some time now :P12:45
blackburnwiking: btw it is not a division/multiplication improvement12:49
wikingit is12:49
blackburnthere are some ifs12:49
blackburnso it couldn't be optimized12:49
blackburnI think gcc optimizes (1/const) things12:49
wikinghow? :)12:50
wikingi mean a division is a division12:50
wikingso if i do a/2 and b/212:50
blackburnat least with ffast-math12:50
wikinghow? :)12:50
blackburnwell now I'm sure12:51
blackburnwiking: yeah there are some -unsafe-math-operations and etc12:52
blackburnit enables to compute x*(1/y) instead of x/y12:52
blackburnthis case contained two 0.5* or /2 but now only one12:53
blackburnthat's what I meant12:54
wikingit should be a bit faster now, but again it should be parallelized13:09
wikingis there a reason not to have openmp in shogun?13:09
blackburnno, feel free to openmp it13:09
blackburnbut you should safe it with #ifdef13:10
wikingyeah sure13:10
wikingthat'll have to do some changes in the configure script as well13:10
blackburnthe problem is kernels are parallelized already13:10
wikinganyhow then i'll do that meanwhile i'm doing the test13:10
wikingin what sense?13:10
blackburne.g. one thread compute one row of kernel matrix13:11
blackburn(if kernel matrix is being computed)13:11
wikingi mean currently when i'm running training13:11
wikingit only runs on one core13:11
wikingso i've figured that in this case the for loop could be sliced up between the number of cores13:12
wikingand just do the calc in separate threads13:12
blackburnIIRC Soeren doesn't mind openmping it as well as I13:12
blackburnsure, makes sense for me too13:12
wikingok i'll then send the pull request when it's ready13:13
blackburnwiking: I would suggest to do following:13:13
blackburnintroduce flag in Kernel.h13:13
blackburnor so13:14
wikingwhy not an ifdef13:14
blackburnand do two branches in each kernel13:14
wikingthat checks a macro13:14
wikingthat is set by the configure script13:14
blackburnhmmm imagine you need to compute whole kernel matrix13:14
blackburnthen it is using 2 cores already13:15
blackburni.e. it runs compute() at the same time13:15
blackburnfor different rows13:15
blackburnand if compute() wants two threads as well13:15
blackburnit would be slower13:15
wikingah got it13:16
blackburnso in computing kernel matrix things we could disable openmp if we are using multiple threads13:16
wikingor of course13:16
blackburnI think double parallelization could be bad13:16
wikingyou could even do better by setting the number of threads openmp allowed to use13:17
wikingand then if you have like 8 cores...13:17
wikinganyhow this is kind of the last thing i wanna test :)))13:17
wikingmore interested in other kernels13:18
blackburnsomething like 2 for kernel matrix and 6 for openmp inside compute()?13:18
wikingand maybe the homkernelmapping..13:18
blackburnkinda complex thing..13:18
blackburnno idea what would be best13:18
wikingyou can tell openmp how many threads it can use in it's threadpool13:18
blackburnyeah I know13:18
blackburnwe have pretty dirty parallelization, should be cleaned indeed13:19
blackburnI'm just not sure whether openmp can handle some complex cases13:20
wikingi'll try to do some test13:21
wikinganyhow let's see the new kernel :)))13:21
blackburnyeah I'm pretty curious13:21
wikingme too13:22
wikingbut it's even worse13:22
wikingcomputing time wise :P13:22
blackburnwhat is the equation?13:22
wikingexp (....)13:22
blackburnworst kernel ever: exp(sin+cos+tg+arctg+exp(exp(exp))13:23
blackburnexplicitly mapping to infinite-dimensional space13:26
wikingmmm fuck13:27
wikingi should test this with custom kernel13:27
blackburnwiking: btw are you still using java?13:34
wikingby data set is kind of coming from mahout/hadoop13:34
wikingand i don't want to now do data transcoding funcitons13:34
wikingit's easier this way13:34
blackburnI see13:34
blackburnI can't live without matplotlib :)13:35
wikingbtw: sum(a^q) * sum (b^q) = sum((a*b)^q)13:35
wikingah no13:35
wikingthat is for sure not true13:36
blackburntrue for the most common case of d=1 :D13:37
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]13:45
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun13:45
n4nd0blackburn, hi13:48
blackburnhi there13:48
n4nd0blackburn, about the ConjugateIndex class13:48
blackburnyeah I'm curious whether this shit works or not13:48
n4nd0blackburn, I have seen one example in the python modular interface using it but it cannot be imported13:48
blackburnmmm do you have lapack?13:48
n4nd0blackburn, I get an error from the interpreter, I didn't find it in the doc. either13:49
n4nd0I don't think I have it because right now I don't even know what is it :P13:49
blackburnremind LPBoost thing, there was an #ifdef HAVE_CPLEX13:49
blackburnhmm you would have to install it :)13:50
blackburnn4nd0: I can't recall which os do u use13:50
n4nd0since I didn't find it in the doc I thought that  maybe the name had changed or sth13:50
blackburnany ubuntu/debian?13:50
n4nd0blackburn, ubuntu13:50
blackburnthen just install everything having name *lapack :)13:51
blackburnand reconfigure13:51
blackburnn4nd0: btw to save time you could also install13:54
blackburnand libarpack13:54
blackburnand libsuperlu3-dev13:54
blackburnand libarpack-dev13:54
blackburnthat enables to achieve better performance while dim.reduction13:54
n4nd0blackburn, thanks :)13:54
n4nd0those will save me some configure and compilation times :D13:54
blackburnno idea whether you would need or not, but you may try at least13:54
blackburnyou may also use --disable-optimizations flag13:55
blackburnin configure13:55
blackburne.g. ./configure --disable-optimizations --interfaces=python_modular13:55
blackburnand do make with -j2 or -j4 (number of threads)13:55
n4nd0number of threads to parallelize the compilation?13:56
blackburnfor example 'make -j4'13:57
n4nd0the disable optimization sounds bad though ... isn't good to optimize :P?13:57
blackburnn4nd0: but it would compile faster then13:59
blackburnyou may see no difference between 1.5s and 1.0s walltime :)14:00
n4nd0it find the ConjugateIndex now :D14:12
wikingit's still counting14:21
wikingjust training i mean14:21
wikingyess! training done14:25
n4nd0blackburn, do you know about this error, it has appeared while training the ConjugateIndex14:25
n4nd0ldc must be >= MAX(M,1): ldc=0 M=0Parameter 14 to routine cblas_dgemm was incorrect14:25
blackburnI know what is the error14:26
blackburncould you please change your labels from -1 and 1 to 0,1?14:27
blackburnit is a bug14:27
n4nd0sure, no problem ;)14:27
n4nd0I have seen in the examples that they use -1 and 1 for two class problems14:28
n4nd0is that the convention in shogun?14:28
blackburnbut conjugateindex is general multiclass14:28
blackburnso I just forgot to implement two 'modes'14:28
blackburnbinary and multiclass14:28
blackburnfor multiclass convention is 0,1,2,.. _without_ any gaps14:29
n4nd0it looks better now, it is taking some time for the training14:29
blackburnyeah it is pretty expensive14:29
n4nd0cool, it trained it, let's see what do I get14:29
n4nd0I have to check a bit of this ConjugateIndex14:30
n4nd0it is completely a black box for me right now14:30
blackburnit computes matrix X^T (X^T X)^-1 X14:32
blackburnfor each class14:32
blackburnX is a class feature matrix14:32
blackburnthen computes x^T (X^T (X^T X)^-1) X) x /<x,x>14:33
blackburnand chooses nearest class for x14:33
blackburnsomething like that14:33
blackburnn4nd0: so how it was?14:35
blackburnI bet it is bad :)14:35
n4nd0blackburn, mmm I am trying to use the class to assess it AccuracyMeasure14:37
n4nd0but I got an error because the labels in the ground truth are 0 or 114:38
n4nd0and it seems that it needs -1 and 114:38
blackburnuse MulticlassAccuracy14:38
blackburnaccuracy measure is for -1, 1 and it compares signs of classes14:39
n4nd0I have tested the accuracy for the training set and it is 0.6714:42
blackburnwhat was other results?14:42
blackburnsvm, knn?14:42
n4nd0this the first one I try :)14:43
n4nd0I am going to check a couple more now14:43
blackburnah okay14:43
n4nd0but this is just using the intensity pixels, no feature14:44
blackburnyes, conjugate thing was designed to cope with pixels14:44
blackburnbut acts badly anyway haha14:44
blackburnwith svm be sure you returned back to -1,114:44
n4nd0what is the use of the cache for the kernels? I have seen this parameter size cache that the constructors have14:47
blackburnto store some k(x_i,x_j) values14:47
blackburnto avoid useless computations14:48
-!- dfrx [] has joined #shogun14:52
blackburnn4nd0: for example when training svms it is very likely that kernels between some support vectors would be calculated many times14:53
-!- dfrx [] has left #shogun []14:53
n4nd0blackburn, I got it ;-)14:56
n4nd0blackburn, some kind of memorization like in DP14:56
blackburnwhat is DP?14:57
n4nd0dynamic programming14:58
n4nd0blackburn, the accuracy with LibSVM is much more promising, 0.9987 (maybe a bit too hight??)15:01
n4nd0blackburn, that is with the training set15:01
n4nd0I have to split it anyway to get a test set15:01
blackburnn4nd0: which kernel did you use?15:03
n4nd0polynomial, degree 215:03
blackburnnot really high, depends on C15:03
n4nd0C = 1.015:03
blackburnit is ok for train set15:03
blackburntry to split it :)15:03
n4nd0I have to go now15:04
n4nd0see you later15:04
blackburnok, see you15:04
-!- n4nd0 [] has quit [Quit: Leaving]15:04
wikinghere's a new pull request15:09
wikingdamn how could i only select 1 commit for the pull request15:11
wikingthere's a bug in HIK15:11
wikingnothing too serious15:12
blackburntell me :)15:12
wikingcheck that15:13
blackburnhaha you scared me15:13
blackburnwiking: just do it in one pull request15:14
wikingi mean yeah15:14
blackburnwhy not?15:14
wikingwhat i could do is that take the optimization + this bug15:14
wikingin one pull request15:14
blackburnit is better to multiply entities ;)15:15
blackburnoccam restricted that15:15
wikingby the way15:20
wikingdo you know BoV?15:20
wikingi guess so15:20
wikingdo you think that it would make a difference that instead a (squared)euclideandistance to use some kind of a different distance measure, like JSdistance? :)15:21
blackburnI am afraid I don't15:21
blackburnwhat is BoV?15:21
wikingbag of visual words15:21
wikingor bow15:22
wikingor bovw15:22
wikingas you feel15:22
wikingso when i'm doing the clustering15:22
wikinge.g. k-means15:22
blackburnI didn't really get into it :(15:22
wikingthe distance measure is dot product atm15:22
wikingbut i think that it's bs15:22
wikingas those sift outputs15:22
wikingare as well histograms15:22
wikingnot just some random values15:22
wikingi'm getting new results15:23
wikingthe JR currently sucks :P15:23
wikingbut we'll see with different arguments15:23
blackburnhmm no idea about distance15:25
blackburnI should dig into BoVW a little15:25
wikingheheh well if u have any questions just shoot15:31
wikingbeen digging in that shit for a while now :P15:31
blackburnwiking: what is a typical measure between sift descriptors?15:39
blackburndistance or similarity or so15:39
blackburnand how does it match points? :)15:39
blackburnI believe answers are in papers, but anyway :)15:41
wikingah there are several way for matching sift descriptors15:50
wikingalthough i'm not doing that at all :))15:50
blackburnthen how do you use any classifiers?15:53
wikingwell i'm using sift just to create visual words16:01
wikingand then from visual words i create a visual words frequency vector16:01
wikingand that's fed to the classifier16:02
blackburnwhat is visual word in that meaning?16:02
wikingwell you get a lot of sift descriptors from a lot of images16:03
wikingand then do an 'averaging' i.e. use a cluster algo16:03
wikingto create visual words16:03
wikingi.e. a codebook16:03
wikingso that the clustering algorithm basically creates N number of clusters16:04
wikingand u assigned each sift descriptor to the nearest cluster point16:04
blackburnok I found some presentation16:06
blackburnexample of word is eye on people photo16:07
blackburnoh it seems I'm getting into it16:07
blackburnfor new unclassified image16:07
blackburnyou compute sift descriptors16:07
blackburnand assign every localized descriptor to nearest cluster16:08
blackburnand then you have e.g. vector [#eyes #cars #bikes]16:08
blackburnwiking: something like that?16:08
wikingmmmnot really16:09
wikingJR is still not good16:09
blackburnok what you are doing once you get descriptors?16:10
blackburnassuming on train set you did clustering of various points of interest16:11
wikingso when u've finished the clustering16:11
wikingu basically assign each sift in an image to a cluster point (the nearest one)16:11
wikingafter this you count how many times a given cluster point is present within an image16:11
blackburneach sift in an unclassified image?16:11
wikingi.e. you'll get a term frequency16:11
blackburnyeah I thought I said the same :)16:12
wikingi.e. cluster centre frequency16:12
wikingnot really :P16:12
wikingbut yeah after you have this TF vector16:12
blackburnwhat is the difference?16:12
wikingyou basically just take the labeling for that tf vector16:12
wikingand learn by that the classifier16:12
blackburnso feature vectors contains rates of various words?16:13
blackburnI feel stupid :D16:13
blackburnis a clustering a mandatory step?16:14
blackburnI mean it could be 'misclustered' :)16:15
wikingthe problem is that otherwise you have just too much different valued sift features16:26
wikingso this is why you create a 'dictionary'16:26
wikingwith clustering16:26
blackburnokay it looks clear now but I can't see my mistake16:27
blackburnwhere I did go wrong?16:27
wikingblackburn: and then you have e.g. vector [#eyes #cars #bikes]16:28
wikingthe clustered descriptors are not telling you anything about the label16:28
blackburn[#rate for cluster 1, ... 2, ...]16:29
wikingso you would have something like16:29
blackburn[0, 3, 5]?16:29
wikingcar: [0,3,5]16:29
wikingif you have a dictionary the size of 3, and this is a dense vector...16:30
blackburnis there any weighting here?16:30
blackburncould it be 1.243?16:30
wikingwell it depends more on your clustering method16:30
wikingbecause you could use16:30
wikingfuzzy clustering16:30
wikingor even do some GMM shit16:30
wikingso it really depends on your approach16:30
blackburnhah I see16:30
blackburnsounds like pretty instable, but working method16:31
wikingwell it works very well16:31
wikingcurrently this gives you the best results16:31
blackburnbut it doesn't fit my task anyway :)16:31
wikinglike on a caltech101 benchmark dataset16:31
blackburnwiking: ok thanks for your explanation16:59
blackburnI'll merge your stuff once I get myself to home:)16:59
wikingcool thnx16:59
wikingmaybe till then i get you a new/better kernel17:00
wikingbut then again it seems the one i thought would be good is not that good17:00
blackburnI can't wait to see results with JS17:02
blackburnheading to gym right now17:09
blackburnsse ya17:09
-!- blackburn [5bdfb203@gateway/web/freenode/ip.] has quit [Quit: Page closed]17:09
-!- blackburn [~qdrgsm@] has joined #shogun20:17
blackburnsonney2k: helloo!20:19
wikinghe's not so sure about my improvement for JS kernel20:32
blackburnyeah I find it strange20:34
wikingwhere should i send things about gsco?20:50
blackburnsend what?20:59
wikingwell about this year21:02
wikingif i want to do soemthing21:02
wikinglike the l-ssvm21:02
blackburnyou would have to wait when shogun is accepted (or not) :)21:05
wikingyeah i know21:07
wikingjust that i have so much other stupid things to do21:08
wikingin case it's accepted21:08
wikingi want to have the whole thingy ready by then :P21:08
wikingi mean proposition etc21:08
wikingnyaah man21:08
wikingthe JR is shit21:08
wikingbut now i'm trying a new one21:08
wikingJT :DD21:08
blackburnman you totally hanged on J* things :)21:09
wikingbecause it's a great thingy21:10
wikingi mean Jensen divergence21:10
wikingit's really good in finding out the similarity between two probability distribution21:11
wikingso yeah of course you want that in CV :P21:11
wikingmaaan this is fucking hilarious:
-!- blackburn [~qdrgsm@] has quit [Ping timeout: 276 seconds]21:40
-!- naywhayare [] has quit [Remote host closed the connection]21:56
-!- blackburn [~qdrgsm@] has joined #shogun22:33
-!- blackburn [~qdrgsm@] has left #shogun []22:50
-!- blackburn [~qdrgsm@] has joined #shogun22:58
wikingblackburn: yo23:40
wikingehehhe no idea23:43
--- Log closed Tue Feb 14 00:00:19 2012