Open in new window / Try shogun cloud
--- Log opened Thu Feb 23 00:00:19 2012
-!- lockless- [] has joined #shogun00:58
-!- lockless- [] has quit [Remote host closed the connection]01:10
-!- Netsplit *.net <-> *.split quits: CIA-6401:22
-!- Netsplit over, joins: CIA-6401:28
-!- dfrx [] has joined #shogun05:49
-!- dfrx [] has quit [Remote host closed the connection]07:24
-!- n4nd0 [] has joined #shogun07:39
-!- dfrx [] has joined #shogun07:57
n4nd0sonne|work: around?08:05
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun11:33
-!- wiking_ [~wiking@huwico/staff/wiking] has joined #shogun11:54
-!- wiking [~wiking@huwico/staff/wiking] has quit [Read error: Connection reset by peer]11:54
-!- wiking_ is now known as wiking11:54
-!- n4nd0 [] has quit [Ping timeout: 245 seconds]12:26
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]13:23
-!- dfrx [] has quit [Quit: Leaving.]13:30
-!- n4nd0 [] has joined #shogun13:53
-!- n4nd0 [] has quit [Read error: Connection reset by peer]13:57
-!- n4nd0 [] has joined #shogun14:02
-!- n4nd0 [] has quit [Client Quit]14:04
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun14:37
sonne|workwiking: now yes :)14:48
wikingsonne|work: i saw that you were publishing some papers with vojtek14:48
wikingand i was wondering if i can ask some stuff about libqp14:49
wikingit's not implementation more just theory14:49
wikingabout the condition of the objective function14:49
sonne|workbtw, vojtech wants to mentor lipqp / shogun integration + some generalizations14:49
wikingheheh would be up for that14:50
wikingi mean i don't know14:50
wikingwhether i can do what i'd like to do now14:50
wikingwith libqp14:50
wikingi've read his paper about it14:50
wikinga novel algorithm for learning ...14:50
wikingdamn i cannot copy-paste from .ps14:51
wikinganyhow the subj. to part is <w, Psi (x_k, y_k) - Psi (x_k, y) >  >= L (y_k, y) - epsilon14:53
wikingand i'm thinking about how to extend it, by having a 3 variable for Psi...14:54
wikingi guess then the problem would not be anymore convex necessarily thus i'd need to do something like CCCP14:54
sonne|workconcatenating variables is not possible?14:55
wikingbut then it'd be quite 'simple' to implement latent-svm14:55
sonne|workahh ok14:55
wikingi want to introduce a latent variable there...14:55
sonne|workdoesnt' exactly sound fast but yes14:55
wikingi've seen this paper Max-Margin Min-Entropy Models from some guys from stanford and i'd like to extend their algo and test it14:56
wikingbut of course the implementation is .... well yeah it's not out anywhere14:56
wikingso i was thinking to do something with shogun14:57
wikinganyhow vojtec is responsive to his emails?14:59
wikingcoz the guy whom i sent about latent-svm never got back to me14:59
sonne|workhe should be, just tell him that you might be interested in his libqp / shogun project14:59
wiking(alexander binder)14:59
sonne|workhe is a nice guy14:59
wikingsent him about a week ago15:00
sonne|workplease send the mail again and CC me15:00
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]15:32
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun15:38
CIA-64shogun: Soeren Sonnenburg master * r0344298 / (2 files): add graphical support vector regression example add myself to svm/svr example authors -
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]16:24
-!- dfrx [] has joined #shogun16:30
-!- dfrx [] has left #shogun []16:34
shogun-buildbotbuild #329 of ruby_modular is complete: Failure [failed test_1]  Build details are at  blamelist: sonne@debian.org17:44
-!- blackburn [~qdrgsm@] has joined #shogun18:29
blackburnsonney2k: sorry things go busy, will clean up gsoc ideas on samstag and sonntag ;)18:32
-!- n4nd0 [] has joined #shogun18:55
blackburnn4nd0: hey, what's up?19:01
n4nd0blackburn: hey! how is it going? :)19:01
blackburnn4nd0: fine, but pretty busy :(19:02
blackburnwhat about you?19:02
n4nd0blackburn: I am fine too, not too many lectures but lot of stuff to do lately :O19:03
n4nd0blackburn: so one doubt I asked soeren but I think he didn't find time to answer19:03
blackburnbut recall I am not an expert in boosting ;)19:03
n4nd0blackburn: yeah sure, it is more about the structure of the projects actually, no about boosting itself19:04
n4nd0blackburn: so I checked the class hierarchy of multiboost and my idea would be first to focus on their algorithm for adaboost19:05
n4nd0blackburn: they have some other stuff apart from that19:05
n4nd0blackburn: so they divide the code of their classifiers in learner and classifier classes19:06
blackburnI see19:07
n4nd0blackburn: those kind of details, should be respected as they are in their implementation19:07
n4nd0blackburn: or adapted to shogun19:07
blackburnwell I guess it won't be very hard to merge it in one class19:07
n4nd0it doesn't sound like a big deal at first sight19:08
blackburnn4nd0: other way could provide some adapter for their code19:08
blackburnI mean you could just insert their code in some multiboost folder19:09
blackburnand the provide classes that use their classes19:09
n4nd0blackburn: that sounds like a good idea19:10
blackburnbtw if you want to do some develop19:11
blackburnI had one idea19:11
n4nd0blackburn: sure, tell me19:11
blackburnwe have no mahalanobis distance19:11
n4nd0blackburn: ok, I have heard of it in a couple of courses :)19:11
blackburnso if you are bored just implement it :)19:12
blackburnok have to go now19:13
n4nd0blackburn: yes! I can do that19:13
n4nd0it sounds better to start with than the boosting thing, it might be to big19:13
n4nd0too big*19:13
blackburnyeah sure I don't think you should do boosting right now19:13
blackburnwill be back later19:14
n4nd0blackburn: ok, thank you for the suggestion!19:14
-!- n4nd0 [] has quit [Quit: leaving]19:50
blackburnsonney2k: hey, around?20:47
@sonney2kblackburn, yes20:47
@sonney2kwriting the gsoc stuff20:47
blackburnsonney2k: hmm exactly what I'm doing20:47
blackburnsonney2k: I strongly suggest to collaborate using docs not git..20:47
blackburngoogle docs I mean20:47
@sonney2kyou just say what you work on and I tell you what I work on20:48
@sonney2kno conflicts no problems20:48
@sonney2kand we are close to done anyway20:48
blackburnok, up to you20:48
blackburnso tell me what you are doing20:48
blackburnsonney2k: will you do application by yourself?20:49
@sonney2kjust polish the one from last year20:50
blackburnI was curious whether  you are going to apply as FML as well?20:50
blackburnehm crazy sentence20:51
@sonney2kno - we will just use shogun toolbox as organization20:51
blackburnsonney2k: ok20:51
@sonney2kthat was confusing everyone last time20:51
@sonney2kblackburn, I just wrote / adjusted kernel two-sample test20:51
blackburnsonney2k: so the only thing I can help is ideas?20:51
blackburnok I'll check it now20:52
@sonney2kmulticlass needs some work and I will do libqp for now20:52
blackburnsonney2k: C5.0 is on?20:52
blackburnhah heiko was just like tolstoy20:53
blackburna lot of text20:53
blackburnsonney2k: what's up with alex binder?20:54
@sonney2kblackburn, what we need now is some more clear structure what we want to do20:54
@sonney2kI mean just 3 major points we want to improve20:54
blackburnI guess it could be result of discussion20:54
@sonney2kaccessibility to shogun (tutorial/ i/o / integration)20:54
@sonney2knew methods20:55
blackburnthat's clear20:55
blackburnto put particular focus on implementen20:56
blackburnhah pretty nice language mix20:56
@sonney2kcurrently it is still a mess20:56
@sonney2kblackburn, please git commit / push20:57
blackburnsonney2k: we have stated ambitious changes again20:57
blackburncommit what?20:57
@sonney2kyour changes20:57
@sonney2kambitious changes?20:57
@sonney2kin what sense?20:57
blackburnstructured outputs20:57
@sonney2kmulticlass/multitask/regression is incremental stuff20:57
blackburnat least20:57
@sonney2kstructured output is not that difficult if you don't do the very difficult algorithms20:58
@sonney2kthe framwork itself is rather easy20:58
blackburnsorry I don't give a fuck how it works20:58
blackburnwell then all is ok20:58
blackburnit is not very clear for me what are the things you want to improve20:59
@sonney2kdid you git push?20:59
blackburnthe only thing I did is added chris' idea20:59
blackburnwhat exactly you want me to push? ;)20:59
@sonney2kyou said you were working on this21:00
@sonney2kif not ok - then I am restructuring completely21:00
blackburnyes I started to make draft application21:00
blackburnbut you said you will21:00
@sonney2kwe have to wait until gsoc interface is open then we have access to the application from last year21:01
@sonney2kideas list is for now most important21:01
blackburnehm you have one in papers folder?21:01
@sonney2kcould be21:01
blackburnI am pretty sure :)21:01
@sonney2kthen I checked it in :)21:01
* sonney2k well done21:01
blackburnhah I thought you navigate better than me here :)21:02
blackburnsonney2k: do you think we need more ideas?21:03
blackburnthe problem is ideas~mentors and we have not much :(21:03
@sonney2kalexander binder, vojtech franc, arthur gretton, chris widmer, oliver stegle, me21:04
@sonney2khow many do you want?21:04
blackburnwell.. no idea21:04
@sonney2kso we can target 6 slots/students21:05
blackburnyes, looks like21:05
blackburnsonney2k: will jmlr reviews be sent to me when it is ready?21:06
@sonney2kyes in 6 months or so :)21:06
blackburnI understand it now :D21:06
blackburnI don't really wait already21:07
blackburnlet it be surprise hah21:07
blackburnsonney2k: I asked vojtech about combining ECOC and the way he did in his 2002 paper21:08
blackburn(genuine MC SVM)21:08
blackburndo you think it is interesting too?21:08
blackburnokay back to ideas21:10
blackburnI've been thinking we have to do a lot of things here21:18
blackburnbuuuut looks like not too much21:18
@sonney2kblackburn, please git pull21:29
@sonney2kand tell me what you think21:29
@sonney2kI think we have everthing in now21:30
@sonney2kmulticlass task should include also the nips papers I mentioned21:30
@sonney2kand we should have some general instructions for students how to apply / what to provide21:30
blackburnsonney2k: do you think examples/tutorial could be a project?21:31
blackburnhave doubts in it21:31
@sonney2kof course21:31
@sonney2klots of programming effort21:31
blackburnhmm may be21:31
@sonney2kand IMHO the most intersting21:31
@sonney2kone gets to know all machine learning algorithms in shogun21:31
@sonney2kwrites examles for them21:31
@sonney2kand generates toy data + some story how these work when they should be used21:32
blackburnokay I'll modify multicalss part21:32
blackburna little later21:32
@sonney2kis the structure now more clear?21:32
blackburnwith tree learning part as well21:32
@sonney2kwith my short summary at the top21:33
@sonney2kimprove in 3 ways...21:33
@sonney2kaccessibility / frameworks / ml algos21:33
@sonney2kI am jsut not sure whether I should put GPs / multitask into framwork or new ML :)21:33
blackburnI don't expect any framework work :) on GPs..21:34
blackburnno idea about multitask yet21:34
blackburnsonney2k: have you ever worked on svms with reject option?21:35
@sonney2kno - is there any difference to multiclass?21:35
blackburnsonney2k: i.e. when you have no examples that should be rejected - you can't do it as multiclass21:37
blackburnso, the only way I find is to threshold (<w,x>+b) < T21:37
@sonney2kbtw, what is your accuracy now?21:38
blackburnmax is still 97.32%21:38
@sonney2kdid you try wikings stuff yet?21:38
blackburnbut linear21:38
@sonney2kso with liblinear?21:38
blackburnnot really21:38
blackburnI didn't manage to use liblinear21:39
blackburnno multiclass :)21:39
@sonney2kwhy don't you do OVR?21:39
blackburnI was working on generic OvR class last weekend21:40
blackburnbut not finished21:40
blackburnbtw vojtech gave me some insight21:40
blackburnwhy genuine mc svm works better21:40
blackburndo you know?21:40
@sonney2kscaling mostly I would guess21:40
blackburnyeah exactly21:40
blackburnI didn't know it21:40
@sonney2kbut it is rare21:40
@sonney2krarely better21:40
blackburn0.3% for me21:40
blackburnI mean GMNP is +0.3% :)21:41
blackburnsonney2k: about wikings stuff - it works well21:41
@sonney2kI guess I should try it too21:41
@sonney2k0.3% is nothing :)21:42
blackburnsonney2k: sure21:42
blackburnsonney2k: I didn't try multiscale hog yet as well21:42
blackburnI expect some improvement too21:43
blackburnsonney2k: liblinear has cramer-singer learning but I have no idea how to attach it21:44
blackburndo you?21:44
@sonney2kwell you have to read their code :)21:44
blackburngood suggestion21:45
blackburnpretty painful :)21:45
@sonney2kand then whenever they access an example call shogun's corresponding dotfeature function21:45
@sonney2kproblem is that they modify data internally (sometimes multiply by label or so)21:45
@sonney2kso this has to be done on-the-fly21:46
@sonney2kblackburn, that is how I did it with liblinear/ocas/libsvm/GMNPSVM/...21:46
blackburnI see21:46
blackburnsonney2k: btw what is better? ocas vs liblinear?21:46
@sonney2kthen in the end compare if the result is the same for same input and if just done :)21:46
blackburnon what?21:46
@sonney2kliblinear is faster but may not converge as nicely21:47
blackburnI have seen there is mc ocas too21:47
@sonney2kand doesn't need that much memory21:47
@sonney2kif you can afford it use mc ocas21:47
blackburnso ocas could converge better?21:47
@sonney2kmuch more robust21:47
@sonney2kblackburn, how many dims do you have?21:47
@sonney2khow many examples?21:48
blackburn39209 train examples21:48
@sonney2kso basically nothing21:48
blackburn2700 dim21:48
@sonney2kthen it doesn't matter21:48
@sonney2kalso nothing21:48
@sonney2kanything will work21:48
blackburnhmm I guess time to try virtuals21:48
@sonney2k(recall that I trained on 50 million examples with ~10 million dims with both)21:49
blackburnno idea how did you do that21:49
blackburnI hardly can wait 30K seconds it trains21:49
@sonney2kfor 40k examples / 10k dims training should take maybe 10s21:50
@sonney2k(except for too large C)21:50
blackburnhmm may be you mean 2 class problem21:50
@sonney2khow many classes do you have?21:51
@sonney2kthen 430 seconds max :)21:51
blackburnor kernel?21:51
@sonney2klinear of course21:52
@sonney2kkernel is slow21:52
blackburnboth GMNP and Larank takes 30K21:52
blackburnyes I didn't manage to do linear yet21:52
@sonney2kblackburn, with larank you could run it in one pass mode though21:56
blackburn how many passes does it do by default?21:56
@sonney2kuntil convergence21:56
blackburnbut why can I do one?21:57
@sonney2kperformance might be owrse21:57
@sonney2kno idea how much ...21:57
blackburnI see21:57
blackburnokay I have to finish generic ovr thing21:58
blackburnno idea how did I broke it :(21:58
blackburnsonney2k: what is COFFIN in shogun?22:00
@sonney2kblackburn, yeah you should really try OVR ....... I would expect it will give very similar results22:05
blackburnah I thought so22:05
@sonney2kvirtual examples have to be programmed independently though...22:05
blackburnI see22:06
@sonney2kbtw I am just upgrading to debian unstable...22:07
@sonney2klets see if shogun still works on this22:07
blackburnwell I use ubuntu 11.1022:08
blackburncan it be considered as debian unstable?22:08
@sonney2kI am a bit scared of gnome 3.x now22:08
blackburnI used to scary unity22:08
blackburnbut works now22:08
blackburnno more butthurt22:08
@sonney2kso you work with unity?22:09
@sonney2kheh so lets see if I can get used to gnome3 then22:10
blackburnI believe it would become ok gradually22:10
-!- n4nd0 [] has joined #shogun22:24
blackburnsonney2k: hah did multiclass thing in python in 15 mins22:30
@sonney2kyeah python makes life soo easy22:31
blackburnmakes mind flow22:31
blackburnbut in C++ makes headache22:31
blackburnsonney2k: ok lets check if your 430s is right :)22:32
@sonney2kliblinear with eps 1e-2 !22:32
-!- blackburn [~qdrgsm@] has quit [Ping timeout: 244 seconds]22:37
-!- blackburn [~qdrgsm@] has joined #shogun22:47
blackburnsonney2k: bad thing about new interfaces like gnome 3 and unity - it eats memory just like pacman22:48
blackburnhuh! it works22:53
blackburnsonney2k: damn you are right!22:56
-!- n4nd0 [] has quit [Read error: Connection reset by peer]23:05
@sonney2kblackburn in which sense?23:07
blackburnsonney2k: in means of train time23:07
blackburnI never thought it is SO fast23:07
@sonney2kand what about accuracy?23:07
blackburnI tried with reduced train set23:08
blackburnwas quite similar23:08
blackburnnow computing with whole train set23:08
@sonney2kI am a bit lost with this gnome 3 thing here ... wonder how I can choose a better theme and get my hotkeys back to work...23:08
blackburnout of memory23:09
blackburnI shall remove reading images as well as hogs23:09
blackburnno idea about gnome 3 :)23:09
blackburnokay will continue tomorrow23:09
blackburnI'll let you know23:09
blackburnif get same 97% with liblinear :)23:10
blackburnsee you23:10
-!- n4nd0 [] has joined #shogun23:10
-!- blackburn [~qdrgsm@] has quit [Ping timeout: 244 seconds]23:14
--- Log closed Fri Feb 24 00:00:19 2012