Open in new window / Try shogun cloud
--- Log opened Wed Jun 20 00:00:41 2012
-!- n4nd0 [53b32c87@gateway/web/freenode/ip.] has quit [Quit: Page closed]00:01
-!- wiking [] has joined #shogun00:33
-!- wiking [] has quit [Changing host]00:33
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun00:33
-!- wiking_ [~wiking@] has joined #shogun03:21
-!- wiking_ [~wiking@] has quit [Changing host]03:22
-!- wiking_ [~wiking@huwico/staff/wiking] has joined #shogun03:22
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 248 seconds]03:23
-!- wiking_ is now known as wiking03:23
-!- wiking_ [~wiking@huwico/staff/wiking] has joined #shogun03:27
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 265 seconds]03:28
-!- wiking_ is now known as wiking03:28
-!- n4nd0 [] has joined #shogun07:09
n4nd0wiking: hey, what's up? how is it going with the PrimalMosekSOSVM example? I read in the logs you were running it07:12
@sonney2kn4nd0 long time no see07:24
@sonney2kI've merged your stuff07:25
@sonney2kInfrastructure  wise I would need to know which functions should return specialized types07:26
n4nd0sonney2k: hi, apply should return an structure type probably07:30
n4nd0not a CStructuredLabels* but a subtype of it, e.g. CMulticlassSOLabels*07:31
n4nd0also the get_label(idx) of CStructuredLabels*07:31
@sonney2kWill you be here in the evening ?07:31
@sonney2kWe can then easily fix those07:32
n4nd0I can do it if you want07:32
n4nd0ok, tell me when and I'll be here07:32
@sonney2kAfter 19 hrs07:32
n4nd0all right, I will be around from 19h then07:33
-!- zxtx [] has quit [Ping timeout: 240 seconds]07:56
-!- nji [31ca5015@gateway/web/freenode/ip.] has joined #shogun08:43
-!- zxtx [] has joined #shogun08:47
njihi!! does anyone know how to use img(erdas imagine raw format) images?08:47
-!- uricamic [~uricamic@2001:718:2:1634:5d58:8730:47cd:6cf8] has joined #shogun08:59
-!- nji [31ca5015@gateway/web/freenode/ip.] has quit [Ping timeout: 245 seconds]09:34
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]09:35
-!- wiking [] has joined #shogun09:37
-!- wiking [] has quit [Changing host]09:37
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun09:37
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]09:50
-!- zxtx [] has quit [Ping timeout: 246 seconds]10:40
-!- n4nd0 [] has quit [Ping timeout: 265 seconds]10:53
-!- zxtx [] has joined #shogun11:34
-!- nicococo [] has joined #shogun12:16
-!- nicococo [] has left #shogun []12:17
-!- wiking [] has joined #shogun12:59
-!- wiking [] has quit [Changing host]12:59
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun12:59
-!- emrecelikten [~emre@] has quit [Quit: Leaving.]13:39
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]14:25
-!- wiking [] has joined #shogun14:48
-!- wiking [] has quit [Changing host]14:48
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun14:48
-!- alexlovesdata [82955843@gateway/web/freenode/ip.] has joined #shogun15:39
alexlovesdatait is only a matter of time when I will fail with these too complex captchas15:40
-!- blackburn [d5578ec7@gateway/web/freenode/ip.] has joined #shogun16:08
-!- blackburn [d5578ec7@gateway/web/freenode/ip.] has quit [Quit: Page closed]16:36
-!- uricamic [~uricamic@2001:718:2:1634:5d58:8730:47cd:6cf8] has quit [Quit: Leaving.]16:43
-!- ckwidmer [8ca3fe9d@gateway/web/freenode/ip.] has joined #shogun17:33
-!- ckwidmer [8ca3fe9d@gateway/web/freenode/ip.] has quit [Ping timeout: 245 seconds]17:58
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]18:21
-!- n4nd0 [] has joined #shogun18:24
n4nd0alexlovesdata: hi, how is it going? do you have a moment?18:26
alexlovesdatayes i have a moment18:28
alexlovesdatadid your MC-SVM got better results after your chat with Nico?18:29
alexlovesdatathe C hack still improves?18:32
n4nd0yeah, with the C hack I am getting results similar to one-vs-rest, what is nice18:33
n4nd0now I am trying to train the SOSVM with bias terms for the multiclass classification example18:34
n4nd0because the % of correct training without bias, about 40%, is too low, and I think it could be achieved even if the implementation is buggy18:34
@sonney2kalexlovesdata, get a real IRC client18:35
n4nd0however, with bias one-vs-rest gets about 90% correct classification18:35
n4nd0so if I manage to get the same result with SOSVM I will be more confident about the implementation18:35
@sonney2khi n4nd018:35
n4nd0sonney2k: hi18:35
@sonney2kn4nd0, I am wondering whether you could compare to some reference18:36
@sonney2klike some CS MC svm18:36
@sonney2kor some matlab code from nico18:36
n4nd0I am comparing to liblinear MC, but it is not SO of course18:36
n4nd0Nico told me that he couldn't find his MC SO code :D18:37
n4nd0I can give it a shot with svm-struct, I have already tried their multiclass test locally18:37
@sonney2kn4nd0, well then nico should just re-do it18:37
@sonney2kit is trivial in matlab...18:37
@sonney2kliblinear MC is not really the same thing18:37
@sonney2kit uses regularized bias...18:38
@sonney2kand it does not really converge too well18:38
@sonney2kbtw, what epsilon did you give liblinear?18:38
alexlovesdatayou are right Soeren18:38
alexlovesdataI will try to get  an irc client18:38
n4nd0sonney2k: epsilon = 1e-318:39
n4nd010e-3 sorry18:39
alexlovesdatasvm struct MC could be well for comparison, as I said yesterday :)18:40
n4nd0alexlovesdata, sonney2k if I want to train my sosvm for MC with bias, should I use regularized bias or not?18:40
alexlovesdatagood question18:42
@sonney2kn4nd0, with 'true' bias18:42
n4nd0sonney2k: I thought that liblinear bias not regularized since the value of the bias I get is much larger than the rest of the components of w, a few orders of magnitude18:42
alexlovesdatawithout regularization would be better18:42
@sonney2kregularized bias is in liblinear because they cannot handly true bias18:43
alexlovesdatabut it is easier to implement with18:43
@sonney2kn4nd0, hmmhh I am not 100% sure with this but the rest of liblinear doesn't have a true bias so I would be surprised if the mc part had18:44
-!- alexlovesdata [82955843@gateway/web/freenode/ip.] has quit [Quit: Page closed]18:44
n4nd0sonney2k: is it the epsilon value relevant? I tend to put it as small as possible (as long as the warnings don't show up) to improve classification results18:45
@sonney2kn4nd0, I just checked the source18:45
@sonney2kthey use regularized bias in MC18:45
@sonney2kn4nd0, yes - the smaller the better18:46
n4nd0can you tell me the line where it appears?18:46
@sonney2kif you compare to mosek - mosek will do 1e-12 or so optimization18:46
n4nd0I tried to check it on my own but lost trace of it18:46
@sonney2kn4nd0, 108 in MulticlassLibLinear.cpp18:46
n4nd0sonney2k: ok, did you want to take a look to the modular issue related to my code?18:49
n4nd0I am surprised anyway sonney2k, look
n4nd0the first two columns are the weight vectors18:50
n4nd0the last column each of the bias18:50
n4nd0epsilon = 10e-5 in this case18:51
@sonney2kn4nd0, what you could do is create some multiclass problem where you put 4 gaussians at the x and y axis at (-1,0), (+1,0), (0,-1), (0,+1)18:51
@sonney2kn4nd0, wait doesn't help18:52
@sonney2kn4nd0, how many classes do you use?18:52
n4nd0yeah, I have thought of that18:52
@sonney2kI mean just use 2 classes18:52
@sonney2kand then doing that would work18:52
n4nd0for more than three classes I cannot come up with any disposition in 2D that makes it linearly separable for hyperplanes without bias18:52
@sonney2kn4nd0, btw why don't you compare to GMNPSVM?18:53
n4nd0I have no idea what is that, let me see18:53
@sonney2kn4nd0, a true multiclass svm. I am not sure about the exact formulation though18:55
n4nd0sonney2k: why do you think it could be better with GMNPSVM?18:55
n4nd0I am taking a fast look and I think it is in the dual18:55
n4nd0I chose LibLinear because I can train in the primal as well, like it is done for the SO-SVM18:56
@sonney2kn4nd0, I dont' know what the current state is18:58
@sonney2kn4nd0, btw disabling bias is probably best then w/ liblinear and in your code18:58
@sonney2kreducing epsilon from 1e-2 ... 1e-6 you should get more and more close results18:59
n4nd0n4nd0: without bias I obtain similar % of correct classification with liblinear and my code, what is good18:59
n4nd0but the thing is that for the example I am trying, 10 Gaussians with 100 samples each quite separated18:59
n4nd0this % is quite low, about 40%-50%18:59
n4nd0I believe there could be an error in my code and still get this rate19:00
n4nd0that's why I wan to try it with bias, LibLinear goes up to 90% with bias for the same example19:00
n4nd0if I get something similar with so-svm then I'll be more confident that the code may be ok19:00
@sonney2kn4nd0, don't compare accuracy19:01
@sonney2kn4nd0, compute the output of each f(x)19:01
n4nd0what should I compare then?19:01
@sonney2kand compare the real valued numbers19:01
n4nd0weight vectors?19:01
@sonney2kn4nd0, normally when you compare convex optimizers you compute the objective value19:02
@sonney2kand if it deviates by not too much all is good19:02
n4nd0yeah I understand that19:03
n4nd0but Nico told me that since they are different solvers for the optimization problem19:03
n4nd0they could be not so equal19:03
@sonney2kn4nd0, so what?19:04
@sonney2kif they are close then it is ok19:04
n4nd0I am testing right now and giving you the results19:04
@sonney2kand results shouldb e more close when epsilon -> 019:04
@sonney2kn4nd0, how can you compute objectives for liblinear?19:04
@sonney2kI guess in your code you have it as output of mosek19:04
n4nd0for liblinear I just do get_w() for each machine19:06
n4nd0aaah! I forgot to say, I am using a MulticlassMachine formed by binary LibLinear19:06
n4nd0not MulticlassLibLinear19:06
n4nd0I should have said that before, sorry19:06
@sonney2kn4nd0, ??19:07
@sonney2khow is this comparable then?19:07
@sonney2kn4nd0, don't you do true multiclass?19:07
n4nd0that's how it is exactly done19:07
@sonney2kn4nd0, well not you should compare it to true multiclass liblinear!19:08
n4nd0aham, why? shouldn't they be more or less the same?19:08
@sonney2kn4nd0, it is like comparing apples and bananas...19:09
@sonney2kn4nd0, really the best thing you can do is compare it to nicos' true MC code19:10
n4nd0:O, really, why? I thought it had actually sense19:10
@sonney2kI mean you need to compare if objectives are the same19:10
@sonney2kclassification accuracy is only a vague indication that things are ok19:11
n4nd0aham, exactly the same optimization problem19:11
@sonney2kas alex said compare it to the true CS thing
n4nd0I am not sure why they shouldn't be equivalent though19:12
@sonney2kso download TJ's svm multiclass19:12
@sonney2kmake sure to disable bias in your code19:12
@sonney2kand to scale C by number of examples in your code (seems like TJ is doing it...)19:13
n4nd0my bias doesn't change anything so ...19:13
@sonney2kn4nd0, it changes objective19:13
@sonney2kand that is all what counts19:13
@sonney2kthen compare objectives19:14
@sonney2kdifference should be *very small*19:14
@sonney2kin the order of epsilon19:14
n4nd0there's no epsilon in my code19:15
@sonney2kn4nd0, in TJs there is19:16
@sonney2kand yours is using mosek's default so should be <1e-1219:16
n4nd0I am not sure this epsilon is mosek's default19:17
n4nd0I have seen that in the paper the bias they use is for the condition to add a new constraint19:17
n4nd0don't read bias ^, read epsilon19:17
n4nd0page 4, algorithm 1, line 819:18
@sonney2kn4nd0, it basically means they tolerate some constraint violation19:21
@sonney2kup to epsilon19:21
@sonney2kso if you set epsilon smaller it will take longer to converge (or will never :D19:21
n4nd0what I meant is that it is not the mosek's default, but another thing19:22
@sonney2kn4nd0, yes of course19:23
n4nd0in my code it will be simply H(y_pred) > xi_i, in line 8 of the algo19:23
n4nd0that's how it is in the code19:23
@sonney2kjust run svm multiclass once with say 0.1 as epsilon19:23
@sonney2kthen compare objectives19:23
@sonney2kthen run it with 1e-4 or so19:23
@sonney2kand compare again19:23
@sonney2kresults shoudl become more similar19:23
@sonney2kwhen decreasing epsilon in svm multiclass19:24
n4nd0my liblinear multiclass right?19:24
n4nd0or should I repeat for MulticlassLiblinear?19:24
@sonney2kdont' care about liblinear19:25
-!- puffin444 [62e3926e@gateway/web/freenode/ip.] has joined #shogun19:25
@sonney2kn4nd0, compare you MCSO svm w/ svm multiclass19:26
@sonney2kpuffin444, hey there .... about your patch - shall we merge it or what is the state19:26
@sonney2kpuffin444, other known issues except for this live user?19:26
puffin444I was hoping that it would be merged, as the additional model selection code would make it a pretty huge patch19:27
@sonney2kpuffin444, well you never said so19:27
@sonney2kso we didn't know19:27
puffin444I have tested it as much as possible for accuracy against GPML and for memory leaks via Valgrind19:27
@sonney2kso only the live user issue is in there right?19:27
@sonney2kpuffin444, so you now use enums to compare classes?19:29
@sonney2kno longer get_name?19:29
puffin444I'm using enums to determine the likelihood models and will do so for other classes for now on.19:29
CIA-18shogun: Soeren Sonnenburg master * r5609bab / (18 files in 3 dirs): Merge pull request #585 from puffin444/master (+11 more commits...) -
@sonney2kpuffin444, btw can we have some example for that?19:31
@sonney2kpuffin444, ahh and one more comment - what do you need iostream for?19:31
puffin444Nothing. I had it in there for debugging and looks like I forgot to take it out. sorry :(19:31
@sonney2kpuffin444, did you already include it in the modular interfaces or are you using it from C?19:31
@sonney2kpuffin444, you can always use SG_PRINT19:32
puffin444Will do so in the future.19:32
@sonney2kand nowadays matrix.display() / vector.display() etc19:32
@sonney2kpuffin444, thanks for the patch19:32
puffin444Right now it's all in C/C++, I can have a python example if you wish.19:33
@sonney2kpuffin444, well some C/C++ one for the moment is OK - later we need some eye candy in python / matplotlib :)19:34
puffin444Absolutely. Would you like me to submit a small patch including an example right away or would if be fine if I just included it when I submit the model selection code?19:34
n4nd0sonney2k: is there something to talk about the modular interface?19:36
@sonney2kn4nd0, yes - you said you wanted to return some specialize code19:37
@sonney2kpuffin444, the earlier we have an example the better - these things get checked on the buildbot so they are kind of a minimal test19:38
@sonney2kn4nd0, err derived class19:38
@sonney2kso I need to know the exact names to get things to work19:38
n4nd0sonney2k: first, since I am using specialized StructuredLabels, e.g. MulticlassSOLabels; I wonder whether apply should return StructuredLabels and later cast or return MulticlassSOLabels directly19:39
n4nd0and then when I do CStructuredLabels::get_label()19:39
n4nd0that returns a CStructuredData*19:39
n4nd0but maybe it would be better to return the derived class, e.g. CRealNumber for the multiclass example19:40
@sonney2kn4nd0, yeah we can do that19:47
@sonney2kbut I need some list for that and an example (in python to see if things work now)19:48
n4nd0sonney2k: things won't work in python now since the swig interface is not fully completed19:52
n4nd0sonney2k: but I can try to do it, make the example work and let you know19:52
n4nd0is that ok?19:53
@sonney2kn4nd0, sure19:54
n4nd0but just as an idea19:55
n4nd0what is the preference, to do swig magic to return correct types or to use things like RealNumber.obtain_from_generic(multiclassSOLabels)19:56
n4nd0however for PrimalMosekSOSVM I doubt if swig magic will work since one may want to return MulticlassSOLabels or other types of labels depending on the application at hand19:56
@sonney2kn4nd0, I would always provide the obtain_from_generic helpers20:04
@sonney2kn4nd0, it is only 'syntactic sugar' to not having to use them from the type unaware interfaces in cases where these are not needed of course20:04
CIA-18shogun: Soeren Sonnenburg master * r140515f / src/interfaces/modular/Machine.i : revert patch deactivating apply magic -
@sonney2kn4nd0, if you want to see how it is done look at interfaces/modular/Machine.i20:11
@sonney2kbasically all I do is some %extend classname { CRegressionLabels* apply(CFeatures* data=NULL) {  return $self->apply_regression(data); } }20:11
@sonney2kand then a %rename of the original apply function20:12
@sonney2kthat's it20:12
CIA-18shogun: Soeren Sonnenburg master * rf4d348f / (43 files in 7 dirs): move kernel normalizers to shogun/kernel/normalizer -
n4nd0sonney2k: ok, thanks20:16
-!- wiking [] has joined #shogun20:18
-!- wiking [] has quit [Changing host]20:18
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun20:18
shogun-buildbotbuild #972 of cmdline_static is complete: Success [build successful]  Build details are at
-!- blackburn [5bde8018@gateway/web/freenode/ip.] has joined #shogun20:19
@sonney2kblackburn, hey there20:20
@sonney2kblackburn, any reason you disabled the apply magic in interfaces/modular/Machine.i20:20
blackburnhmm did I?20:20
@sonney2kall commented20:21
@sonney2kI fixed that in 140515f139e1dd9605d96ea28a80d1bfd8e92a2d20:21
blackburnI do not remember I did that20:22
@sonney2kblackburn, git blamed you20:22
@sonney2kand git never lies20:22
-!- blackburn [5bde8018@gateway/web/freenode/ip.] has quit [Ping timeout: 245 seconds]20:26
-!- blackburn [5bde8018@gateway/web/freenode/ip.] has joined #shogun20:30
blackburnsonney2k: I do not really understand why it became commented O_o20:30
blackburnit was rather unintentional20:31
shogun-buildbotbuild #973 of cmdline_static is complete: Failure [failed test_1]  Build details are at  blamelist:, sonne@debian.org20:36
-!- blackburn [5bde8018@gateway/web/freenode/ip.] has quit [Quit: Page closed]20:42
@sonney2kpuffin444, can you please have a look at examples/undocumented/libshogun/regression_gaussian_process.cpp ?20:55
@sonney2kit fails currently...20:55
@sonney2kfails to compile that is20:55
puffin444Oh No. It looks like I forgot to update the regression_gaussian_process.cpp file for the new interface :-/20:56
puffin444I am uploading a new example ASAP.20:58
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]21:03
-!- puffin444 [62e3926e@gateway/web/freenode/ip.] has quit [Ping timeout: 245 seconds]21:17
-!- puffin444 [62e3926e@gateway/web/freenode/ip.] has joined #shogun21:18
-!- wiking [] has joined #shogun21:39
-!- wiking [] has quit [Changing host]21:39
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun21:39
@sonney2kblackburn, chris' patch breaks stuff - examples in libshogun won't work21:51
CIA-18shogun: puffin444 master * r0051549 / (4 files in 3 dirs): Updated Gaussian Process Regression example. -
CIA-18shogun: Soeren Sonnenburg master * rcf381d6 / (4 files in 3 dirs): Merge pull request #591 from puffin444/master -
@sonney2kpuffin444, btw you can use init_shogun_with_defaults()21:56
@sonney2kthen you don't need to set a print etc function21:56
puffin444I'll quickly change the example if you want with that.21:57
puffin444Sorry about breaking the build.21:57
@sonney2kpuffin444, blackburn/chris broke it before you so relax :)22:00
@sonney2kand they are both hiding22:00
@sonney2kwiking, any news on your PR?22:00
@sonney2kwiking, a real one tomorrow?22:12
wikingi have a merging error here22:13
wikingtrying to resolve22:13
wikingsomething happened with the Makefile in the example22:13
wikingthat i cannot merge in22:13
puffin444sonney2k, are we still using NLOPT in shogun?22:17
@sonney2kpuffin444, if you need it - you can. pluskid found another lib that seems to be better suited for his optimization problem22:18
@sonney2k(see his pull request - maybe it fits to what you want to do)22:18
puffin444sonney2k, I saw pluskid's patch and I wondered if it would be easier just to use his lib22:19
@sonney2kpuffin444, look at it!22:19
@sonney2kit seemed very promising to me22:19
puffin444It does to me too. My only concern is that it doesn't support constraints22:20
puffin444But I think the only constraints I have are that the hyperparameters must be positive.22:20
@sonney2kpuffin444, try it out on some toy example - I would bet it supports box constraints...22:22
puffin444Would optimizing over log space work?22:22
@sonney2kpuffin444, no idea - have a look at this lib (I don't know it either) or ask oliver22:23
puffin444I think it would - Oliver mentioned something like this.22:24
@sonney2kIIRC will a log transform keep things convex22:28
@sonney2kactually log(x) is a counter example - haha22:30
n4nd0puffin444: about optimizing after applying log, it depends on what are you applying log to I think22:35
n4nd0depending on its convexity/concavity and if it's decreasing/increasing it can turn the problem into convex and so22:36
-!- n4nd0 [] has quit [Ping timeout: 264 seconds]22:44
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]22:46
-!- emrecelikten [~emre@] has joined #shogun22:53
-!- puffin444 [62e3926e@gateway/web/freenode/ip.] has quit [Quit: Page closed]23:22
-!- wiking [] has joined #shogun23:47
-!- wiking [] has quit [Changing host]23:47
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun23:47
--- Log closed Thu Jun 21 00:00:41 2012