Open in new window / Try shogun cloud
--- Log opened Sun Jun 17 00:00:41 2012
blackburnOk finally first multitask regressor read00:04
CIA-18shogun: Sergey Lisitsyn master * rf621fdf / (14 files in 4 dirs): Updated TaskGroup, added train routines for MultitaskLS regression and fixed some bugs in SLEP ported code -
CIA-18shogun: Sergey Lisitsyn master * rafe730c / (14 files in 4 dirs): Merged SLEP updates -
blackburnn4nd0: I'm now porting algo that does SVD each iteration (sic!)01:12
n4nd0blackburn: and how is it going?01:15
blackburngoing ok but SVD!01:15
blackburneach iteration!01:15
n4nd0too much cost right?01:16
shogun-buildbot_build #632 of octave_modular is complete: Success [build successful]  Build details are at
blackburnwell the only exciting thing about svd is its robustness01:17
blackburnwith cond=1 it is very stable01:18
-!- emrecelikten [~emre@] has joined #shogun01:32
shogun-buildbot_build #608 of lua_modular is complete: Failure [failed test_1]  Build details are at  blamelist: blackburn91@gmail.com02:06
shogun-buildbot_build #633 of octave_modular is complete: Failure [failed compile]  Build details are at  blamelist: blackburn91@gmail.com02:12
-!- blackburn [~blackburn@] has quit [Ping timeout: 244 seconds]02:25
-!- n4nd0 [] has quit [Ping timeout: 246 seconds]02:44
-!- emrecelikten [~emre@] has quit [Quit: Leaving.]06:35
-!- emrecelikten [~emre@] has joined #shogun08:05
-!- emrecelikten [~emre@] has quit [Quit: Leaving.]09:33
-!- n4nd0 [] has joined #shogun10:35
-!- blackburn [~blackburn@] has joined #shogun10:52
-!- n4nd0 [] has quit [Ping timeout: 240 seconds]11:29
-!- blackburn [~blackburn@] has quit [Ping timeout: 248 seconds]12:13
-!- n4nd0 [] has joined #shogun12:39
-!- n4nd0 [] has quit [Quit: Lost terminal]13:53
-!- emrecelikten [~emre@] has joined #shogun17:42
-!- blackburn [~blackburn@] has joined #shogun18:37
-!- n4nd0 [] has joined #shogun19:42
blackburnn4nd0: hey any progress with your bug?19:48
n4nd0blackburn: I changed a couple of things and the results improved, but I don't find satisfactory yet20:02
blackburnI see20:05
n4nd0blackburn: but the way, let me ask you something20:07
n4nd0general SVM theory :D20:07
blackburnI've been waiting for this moment whole my life ;)20:07
n4nd0I have seen that in the paper they write20:08
n4nd0min_w,\xi 1/2 ||w||^2 + C/n sum_i \xi_i20:08
n4nd0if you prefer to look at it ^20:08
blackburnso the question is C/n?20:09
n4nd0page 2, equation 1, near to the end of the first column20:09
n4nd0it is 1/n in particular20:09
blackburnyes found20:09
n4nd0I am not using that currently20:09
n4nd0I don't know if it should make a difference20:09
blackburnit doesn't make difference - just changes C implicitly20:10
blackburnwhat changes the game is \sum_i C_i \xi_i20:10
n4nd0why C_i?20:10
blackburnit can be done with differents C's20:11
n4nd0one for each example20:11
n4nd0I've never seen that, but it's ok20:11
blackburnbut while you just play with constant before the sum20:11
blackburnI don't think it makes and difference20:11
blackburnI think it is easier for they to prove things with 1/n20:12
n4nd0I am going to use how liblinear does it internally20:13
blackburnn4nd0: does what?20:16
n4nd0use 1/n20:16
blackburnthey do not use 1/n20:16
blackburnI am 99% sure they don't20:16
blackburnn4nd0: because finley and joachims introduced that I think :)20:17
n4nd0the 1/n20:18
blackburnstandart C-S does not involve 1/n20:18
blackburnneither W-W svm does20:18
blackburnn4nd0: liblinears multiclass svm is just kind of gradient descent with cooling20:20
blackburnsimply they just exclude alphas that zeroed20:21
blackburnand work only with subset of alphas20:21
blackburnocas does cutting plane btw20:22
@sonney2kn4nd0, the 1/n is kind of a relegion20:30
@sonney2ksome use it some don't20:30
@sonney2kn4nd0, rationale is that the \sum_i \xi_i is an upper bound on the training error20:31
@sonney2kif you divide by n it is the risk aka fraction of misclassified examples20:32
@sonney2kbringing that on the same scale as the ||w||_2^2 makes some sense20:32
@sonney2khowever, I find it not very intuitive in practice at least:20:33
@sonney2kC does not scale linearly with n20:33
@sonney2kn4nd0, btw what is up with your pull request? you never answered my comment...20:36
n4nd0sonney2k: yeah sorry, since there are still some things that I have to correct, apart from the renaming and re-allocating of the files you told me, I thought it was clear that it was not a definitive version to merge yet20:39
n4nd0I don't know if you have heard about my fight against the bugs during this week ...20:40
n4nd0I have not really coded anything new this week, I am just testing, debugging, testing, debugging, etc20:40
blackburnsonney2k: mldata is not so cool! I never managed to find some popular datasets here :(20:40
n4nd0sonney2k: btw, you never answered either my mail on the mailing list about the issue with the modular interface :(20:41
blackburndid somebody review puffin444?20:45
blackburnI did20:53
@sonney2kn4nd0, it does not have to be ready as in everything is 100% - it needs to compile and uricamic/wiking can work with the infrastructure20:59
@sonney2kn4nd0, so it is really highly needed20:59
@sonney2kblackburn, what dataset is popular/missing?21:00
@sonney2kblackburn, mldata just needs more users/uploaders...21:00
blackburnsonney2k: today I did not manage to find 'school' dataset21:00
blackburnfor multitask21:00
@sonney2knever heard of it21:00
@sonney2kso upload it then :D21:01
blackburnyeah if I had21:01
blackburnsonney2k: they compare on school dataset in every damned multitask paper21:01
n4nd0sonney2k: what I commited and introduced in the pull request compiles and doesn't crash anything21:34
n4nd0sonney2k: but I will introduce the changes you told me aobut asap so you can merge21:35
n4nd0sonney2k: anyway, I am sorry to ask you again, did you have time to take a look to the mail I sent about the modular interface?21:35
n4nd0blackburn: I am starting to be happy with my results :)22:00
n4nd0blackburn: at least they are similar on average with the ones given by multiclass liblinear not using bias22:01
n4nd0blackburn: around?22:11
blackburnn4nd0: yes22:36
blackburnn4nd0: that's cool22:37
n4nd0blackburn: I think I am going to include the possibility to include bias in LinearSOMachine22:37
blackburnregularized bias?22:37
n4nd0blackburn: if the results of multiclass SO improved more (as it does with LibLienar) then I will be more or less convinced that the impl. is ok22:38
n4nd0blackburn: I was thinking of including bias, not regularization as this moment22:38
n4nd0blackburn: I think that the way it's done in SO, regularization would be included using the matrix C22:39
n4nd0min w'*C*w22:39
blackburnn4nd0: is it possible to include bias in that kind of formulation?22:39
n4nd0why not?22:39
blackburnwell how?22:40
n4nd0adding one more dimension to w22:41
n4nd0and the same for the feature vectors22:41
n4nd0the one in w is to be learnt and in the feature vectors fixed to 122:42
blackburnn4nd0: it is regularized bias ;)22:43
n4nd0where is the regularization there?22:43
blackburnit goes to regularization term22:43
blackburnso when you optimize w you try to keep b low too22:44
blackburnit is not bad (Mangasarian, 1992) but it is22:44
blackburnjust is :D22:44
n4nd0is what?22:44
blackburnit is regularized I mean :)22:44
blackburnactually our dear vojta did that in his fancy svm22:45
blackburnGMNP one with Kesler transformation and a lot of other stuff22:45
blackburnn4nd0: haven't you been watching por-ned?22:46
n4nd0blackburn: they showed germany-denmark here, not por-ned22:46
blackburntoo bad22:46
blackburnronaldo was fscking awesome22:47
n4nd0he's fucking awesome :)22:47
blackburnportugal has very strong sprinter skills I am amazed with22:47
blackburnI bet they do these 50-60 meters in 6-7 seconds22:48
n4nd0Greece-Germany in quarter finals22:49
n4nd0funny fact :D22:49
blackburnRussia-Germany would be funnier22:49
n4nd0no way22:49
blackburn22th June22:49
blackburnthe day war began22:49
n4nd0Greece - Germany with everything about the economic problems, seems fair enough to me22:49
blackburnheh well war is more important I believe :D22:50
blackburn22th June, 194122:50
n4nd0aren't we talking about an old war dude?22:50
n4nd0some time has passed ever since22:50
blackburnvery important still for russia and germany too I think22:51
blackburnin the end we have lost ~20M22:52
blackburna little less than half of spain :D22:53
n4nd0Spain has nothing to do with this ;)22:53
blackburnand two swedens22:53
blackburndid you grow?22:54
n4nd0no ... I'm still getting shaved :)22:54
blackburnnow I do understand why I have not so outstanding skills22:54
n4nd0Captain Soeren was at Guru level in this moment :D22:55
blackburnyeah right22:56
blackburnwhy do he call himself Kristian :D22:57
blackburnin that video he is talking on behalf of some Kristian22:58
n4nd0aha :)22:58
n4nd0good night!23:00
-!- n4nd0 [] has quit [Quit: leaving]23:01
--- Log closed Mon Jun 18 00:00:41 2012