gsomix alexlovesdata alexlovesdata --- Log opened Wed Jun 06 00:00:41 2012 -!- heiko [~heiko@host86-179-59-69.range86-179.btcentralplus.com] has quit [Ping timeout: 256 seconds] 01:07 -!- blackburn [d5578aee@gateway/web/freenode/ip.213.87.138.238] has quit [Ping timeout: 245 seconds] 02:39 -!- romi_ [~mizobe@187.66.121.115] has quit [Quit: Leaving] 04:36 -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Read error: Operation timed out] 05:47 -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] 06:07 -!- wiking [~wiking@78-23-189-112.access.telenet.be] has joined #shogun 06:13 -!- wiking [~wiking@78-23-189-112.access.telenet.be] has quit [Changing host] 06:13 -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun 06:13 -!- uricamic [~uricamic@2001:718:2:1634:29b5:2f5b:6ebd:d1b0] has joined #shogun 09:01 -!- gsomix [~gsomix@109.169.142.23] has joined #shogun 10:01 -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] 10:05 hi 10:08 -!- wiking [~wiking@we02c096.ugent.be] has joined #shogun 10:34 -!- wiking [~wiking@we02c096.ugent.be] has quit [Changing host] 10:34 -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun 10:34 -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun 11:08 -!- gsomix [~gsomix@109.169.142.23] has quit [Quit: Ex-Chat] 11:18 -!- gsomix [~gsomix@109.169.142.23] has joined #shogun 11:20 -!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 265 seconds] 11:52 -!- alexlovesdata [82955843@gateway/web/freenode/ip.130.149.88.67] has joined #shogun 12:00 may I ask: who is zxtx, naywhayare, and the CIA agent ? 12:07 with respect to the others I have an idea who they are 12:08 fan of the software 12:14 working on a patch to get pegasos into the repo 12:14 ahh thx! 12:15 alexlovesdata, CIA-9 is github bot. 12:15 thx 12:17 -!- wiking [~wiking@78-23-189-112.access.telenet.be] has joined #shogun 12:23 -!- wiking [~wiking@78-23-189-112.access.telenet.be] has quit [Changing host] 12:23 -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun 12:23 -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving] 12:58 -!- alexlovesdata [82955843@gateway/web/freenode/ip.130.149.88.67] has quit [Ping timeout: 245 seconds] 13:11 -!- alexlovesdata [82955843@gateway/web/freenode/ip.130.149.88.67] has joined #shogun 15:26 -!- puffin444 [472e31fb@gateway/web/freenode/ip.71.46.49.251] has joined #shogun 16:05 viking? 16:11 alexlovesdata: yeps here 16:25 nice 16:26 ok so what you would like to do next? 16:26 alexlovesdata: so yeah as i was writing to you earlier the very short plan for the latent extension of shogun would be to get a simple -1,1 labelled data and solve for that the optimization problem. currently i have a dataset with mammal's in it. and the 'task' is basically to label the image (i.e. give the right mammal present in the image) and give a bounding box for it 16:27 so joint object detection and classification 16:27 so this is a very typical example for object recognition in images, i.e. your h would be something like (x,y) and (w, h) 16:28 caltech256 animals? a dataset which I had used 16:28 then yuo could cite me :) 16:28 hheheheh :) 16:28 unofficial Gsoc rule: cite your mentor, joke aside: 16:28 imho there's only 6 different mammals in this dataset 16:28 caltech256 animals has 52 or so 16:29 i cannot recall now which dataset is this 16:29 except for the fantasy animals like minotaur 16:29 but then again it's small and simple... 16:29 and the features are already ready 16:29 thats not important 16:29 so yeah anyhow... i was trying now that example to get working 16:29 so you want to implement the felzenszwalb style latent svm> 16:30 yes 16:30 so you want to implement the felzenszwalb style latent svm? 16:30 so first that one 16:30 and after that try to work on a SO latent svm 16:30 as soon as i can start using the solver of n4nd0 (fernando) 16:31 and psi(x,h) would be = ? HoG feature over the box? 16:31 yep 16:31 so now i have like n hog for each image 16:31 ok, sounds easy 16:31 my suggestion would be to get a base latentpsi class and then derive your special class from it\ 16:32 which has its own argmax 16:33 and its own way to get the psi(x,h) 16:33 ah so that one can already use this 'example' for other object recognition 16:33 what do you mean by your last question? 16:34 so i mean that we have this derived class 16:34 yes 16:34 which users can use out of box 16:34 if they wanna have an object detector 16:34 for exmaple.. 16:35 ok 16:35 yes, good idea 16:35 that should be fine, although i have to discuss about this with blackburn (sergey) 16:35 since i think it should be part of the library itself and not the example part in the repository 16:36 but this is some minor thing 16:36 what to discuss? 16:36 well where exactly to store the code for this 'example' 16:36 anyhow i was just wondering if there's such easy example for latent svm 16:36 that would be a usual use case 16:36 as it would be good to have 2-3 use cases for latent svm implemented in the library 16:37 in examples/someinterface ? 16:41 examples/undocumented/* 16:41 or do you mean C++ code? 16:41 C++ code you can have as test method even 16:42 alexlovesdata: i mean that i think these basic 'examples' should actually be really part of the library itself. so that one could just basically include in his own code an ObjectDetector.h or something which is basically a latent svm based object recognizer... 16:43 yes, this is ok 16:44 you have a base class and a derived example 16:44 yep 16:44 but i need to talk about this with blackburn... how exactly we should do this 16:44 i mean where to put the actual code/header... 16:44 and then some code for interfaces  (python whatever) 16:45 yes put it into shogun main 16:45 yeah the modular interfaces will be the last step ... 16:45 so first only c++ and then when it all works fine i'll do the modular interfaces for python etc.. 16:45 because it is a usable piece of code 16:45 my question would be 16:45 will you also implement the mining of hard negatives 16:47 aaaah 16:48 not this week :D 16:48 no not this week 16:48 I wanted to say: it is NOT mandatory for latent SVM 16:48 but yeah i was thinking about it 16:48 I think it is not your core duty to implement Felszenszwalb in all details, ok? 16:49 so if you do binary latent SVM fine 16:49 doing more like hard negatives mining is NOT mandatory. 16:50 everything besides mining hard negatives is luxury ... for donald Trumps wife 16:50 yea but actually it would be great to have imho 16:50 and of course when SO is in a working shape, it'd be great to have latent structural svm 16:51 what i want this week is really the simple solver i've mentioned earlier.... based on ocas 16:53 thats fine! 16:53 and pls do not waste time on more than mining hard negatives 16:54 :> 16:55 will try :) 16:56 hmm, should we discuss nandos struct while he is not in chat? 16:57 bash it and talk about our wishes :) ? 16:58 :D 17:00 https://github.com/iglesias/shogun/tree/master/src/shogun/so 17:00 ahhahaha 17:00 because if we discuss this is three weeks it might be too late 17:00 so now is time for wishing what we want 17:00 yeah i've already told him 2 weeks ago 17:01 w 17:01 what i want :D 17:01 any desired changes which you could tell me ? 17:01 so that I know what you want, too :D ? 17:01 well i think the problem here will be 17:01 that i'll have basically 2 base classes 17:02 1) a base class latent svm solver with -1,1 labelling 17:02 2) same but with structured labelling 17:02 i don't see it being able to cover by only 1 base class 17:02 so lets say i'll have something like: LatentLinearMachine and LatentStructuredLinearMachine 17:03 does that affect nandos framework? because for -1,+1 labels you could use ocas and fine 17:04 am i wrong? 17:04 wait I get myself a coffee for 3 minutes 17:05 no you are wrong 17:05 ok no worries 17:05 i'll write the rest here in the meanwhile... so afaik LatentStructuredLinearMachine can be derived from CLinearStructuredOutputMachine 17:05 and that would be the latent s-svm solver 17:06 -!- romi_ [~mizobe@187.66.121.115] has joined #shogun 17:09 if I am wrong then pls correct me 17:12 back from getting coffee 17:12 ok 17:13 -!- uricamic [~uricamic@2001:718:2:1634:29b5:2f5b:6ebd:d1b0] has quit [Quit: Leaving.] 17:15 afaik LatentStructuredLinearMachine can be derived from CLinearStructuredOutputMachine ... 17:20 I would say: you use it rather as a solver instead of deriving 17:20 so it would be a member 17:20 or called in train() 17:20 or called in train() only 17:20 that might be easier than deriving it 17:21 then you can use nandos stuff only as a solver and are free to design your own interfaces as you like them most 17:21 mmm 17:21 but we'll have to change it 17:22 since the PSI is different in case of a 17:22 LatentStructuredLinearMachine and CLinearStructuredOutputMachine 17:22 PSI(x,y,h) vs PSI(x,y) 17:22 so i wouldn't be able to use it directly 17:23 right 17:23 for solver calls you know the h's already 17:23 so you could add to your member a getpsi_knowhiddenlabels 17:24 method 17:24 which inputs the right psi into nandos olver 17:24 wrong? 17:24 mmm 17:25 if you see a problem in it pls say so ... mistakes belong to me like rotten fruits to a market 17:25 * wiking thinking 17:25 -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun 17:26 so this is the current implementation of an SO solver: https://github.com/iglesias/shogun/blob/master/src/shogun/so/VanillaStructuredOutputMachine.cpp 17:27 this as is i won't be able to use as a solver directly in CLinearStructuredOutputMachine 17:27 i've meant: LatentStructuredLinearMachine 17:27 or i don't see it yet 17:28 mmm 17:29 ok i'll be able 17:29 -!- blackburn [d5578d64@gateway/web/freenode/ip.213.87.141.100] has joined #shogun 17:29 i'll only need to change the CStructuredModel 17:29 hey 17:29 I just read logs - wiking what is the code you want to discuss where to put it in? 17:29 because in https://github.com/iglesias/shogun/blob/master/src/shogun/so/VanillaStructuredOutputMachine.cpp#L34 the passed CFeatures* data would be actually the already calculated PSI(x,y,h) 17:30 blackburn: ok so let's say i have a basic latent svm solver, e.g. LatentLinearMachine 17:30 right 17:30 that needs a lot of parameters/functions implemented if you actually want to use it for an actual problem 17:30 so let's say u want to have an object detector in an image based on latent svm (typical use case) 17:31 that would be something like ObjectDetector : public LatentLinearMachine 17:31 wiking: there have been some changes in Vanilla and other classes 17:31 wiking: check my branch so to see the latest ones 17:31 imho that class is so 'often' used that actually having it in shogun library would be good 17:31 yes 17:32 n4nd0: https://github.com/iglesias/shogun/blob/master/src/shogun/so/VanillaStructuredOutputMachine.cpp#L34 17:32 I don't mind to put it into shogun/latent 17:32 isn't this the latest? 17:32 blackburn: ok 17:32 wiking: no 17:32 n4nd0: ? 17:32 :D 17:32 where is it then? :D 17:32 however there could be some issues 17:33 i.e if you want HoG there 17:33 wiking: in the branch 17:33 wiking: not in master 17:33 n4nd0: oh shit yeah sorry :DDD 17:33 no problem :D 17:33 blackburn: well that implementation is not dependent on HoG itself 17:33 so you could use other features 17:33 I have a bunch of new changes too I will push soon them to the branch 17:34 right, all what we need is a class for psi(x,y,h) 17:34 wiking: then it could become pretty big 17:34 can anyone give me the link to the relevanr branch? 17:34 I don't mind to put it into applications as well 17:34 alexlovesdata: https://github.com/iglesias/shogun/tree/so/src/shogun/so 17:34 n4nd0: liked the other api better :))) 17:37 n4nd0: was more flexible :P 17:37 wiking: because of the function pointers? 17:37 not just because of that 17:37 right, this api is a bit more special 17:37 because it separates the structures labels from the features 17:38 in SO this split is artificial 17:38 one works over Psi(x,y) 17:38 wiking, alexlovesdata : tell me what parts you don't like and we can adapt it 17:38 this can be constructed from phi(x) and y 17:38 but that is not necessary ... 17:38 n4nd0: i'm just checking https://github.com/iglesias/shogun/blob/so/src/shogun/so/StructuredModel.h 17:39 as basically that's the thing i'll have to modify 17:39 or create a derived class 17:39 if I am allowed to say something ... 17:39 what do you want to modify? 17:39 alexlovesdata: sure :) 17:39 go ahead 17:39 if we would habe an alternate setter which allows to input the Psi(x,y) directly 17:39 without constructing them from struct labels and features 17:40 for our stuff we will probably use a psi class to get this abstraction ... I do not require this 17:40 alexlovesdata: the part of Psi is quite undone so far 17:40 but as thinking input it might be an idea 17:40 the psi class has its own argmax 17:41 depending on the structure 17:41 and can be initialized as one likes 17:41 eg explicitly by set structlables and set features 17:41 the point is: the struct solver needs only psi(x,y) and information which x and which y belongs to each psi and the set of all possible ys and x's 17:42 everything else is more specialized to some applications 17:42 am I wrong?? 17:42 thats why I would like to have a way such that one can input the psis directly together with an argmax 17:43 and that was possible with the old C-style interface 17:44 by overriding the function pointer 17:44 but you can do that with the new interface as well 17:44 please prefer interfaces 17:44 alexlovesdata: my idea is that Psi would be a class member of the StructuredModel 17:44 ... with a different abstraction however 17:44 pointers is more painful for modular interfaces 17:44 alexlovesdata: then you could have a set_psi there too 17:45 I agree blackburn 17:45 and its uglier 17:45 with brand new directors (TM) we can do some funky shit here 17:45 however even set_psi is bad when the psis are too big for memory 17:45 so it would be better to have a psi class which has its get_a_specific_psi member 17:46 because the solver could just use the getter to get the right psi 17:46 and its associated struct label 17:46 and no need for members like vector 17:47 we should also take into account that sonney2k wants to have the joint features or psi with the idea of COFFIN 17:47 so I think that at the end we will use a class similar to CDotFeatures 17:47 well, the psis need then a scalar prod (member of class) and a linadd ... 17:47 yep 17:47 I have not looked into CDoTFeatures but my idea behind it is: it needs then some getter for the psi(x_i,y_i) 17:48 and for its associated label index and feature index 17:48 that should be enough for the solver 17:49 and a derivied class could implement a psi from Cfeatures and Cstructlabels ... as done now in the current code 17:49 so you would retain the current functionality 17:50 just split the solver from getting the psis 17:50 thats my suggestion ... sorry for assholing around 17:50 so the structuredmodel would have a setpsiclass member or so 17:51 -!- puffin444 [472e31fb@gateway/web/freenode/ip.71.46.49.251] has quit [Quit: Page closed] 17:51 is that an idea?> 17:52 n4nd0: ping :) 17:52 alexlovesdata: so what you suggest is to have a psi_function that is a member of the model with a setter 17:53 or? 17:53 yes 17:53 but psifunction is a class itself 17:53 wiking: I was answering, it takes some time to read and think :P 17:53 n4nd0: :>>> no worries 17:53 alexlovesdata: I agree with that suggestion, it's the idea I have 17:54 I get older, too ... 17:54 ok 17:54 I have not so clear though what functionality we should provide in this base psi_function class 17:54 -!- puffin444 [472e31fb@gateway/web/freenode/ip.71.46.49.251] has joined #shogun 17:54 what the solver needs ... 17:55 1)accessing the i-th psi 17:55 getting its index into training data (no x_i actually  necessary) 17:55 getting its index into structlabels 17:55 if we assume that structlabels are discrete 17:56 where i-th represents both an index for a feature and another for a label? 17:56 the index could be for a continuous case also a vectro of real numbers 17:56 but in the simplest discrete case it is a long or a vector of long 17:56 for prediction it needs the range of struct labels 17:57 not for the solver: ways to construct these psis 17:57 index for training data means the i for which trainning data point x_i 17:58 index for structlabels means: which y was used 17:59 so these indices are two different things 17:59 so these two indices are two different things 17:59 I need a black tea for a moment 18:00 back 18:03 ok, tell me then 18:04 nando: you could check for a struct formulation what it needs besides the Psis 18:04 then you know what members you will need for the psi class 18:04 it will also have its own argmax 18:04 because that depends on the structure of Psi 18:04 why? 18:04 I think that Psi and ArgMax should be different parts 18:05 I don't understand why the psi function have its own argmax 18:05 because  max_{y \in Y}  w*psi(x,y) 18:06 depends on the structure of psi(x,y) and y 18:06 in the most generic case it would be searching all y's brute force 18:06 in more special cases you would search only some y's based on their structure 18:07 eg in computer vision only some bounding boxes close to a given one 18:07 y=bounding box params 18:07 thats why argmax would be a mamber of the psi class 18:07 wrong? 18:07 I understand your point 18:08 but thinking of the code, it looks to me kind of weird that the psi function has its own argmax 18:08 it is like, the psi function is computed independently of how the argmax is computed 18:09 then, why argmax should me a member of psi? 18:09 because I would say that computing the argmax depends on the structure of psi and y 18:09 -!- romi_ [~mizobe@187.66.121.115] has quit [Ping timeout: 244 seconds] 18:10 I think a good starting point would be if you look into the SO formulation based on: how wouldwe start if we would load psi(x,y) from disk 18:10 what member would the psi class need 18:11 psi class needs labels and features 18:11 if you work directly with precomputed psis 18:11 only labels, no x's 18:11 why not? 18:11 because in SO SVM you never use the X directly in optimization 18:11 only Psi(x,y) 18:12 am I worng? 18:12 am I wrong? 18:12 ok, so you mean like we use a particular example of X (let's sat an x_i) but not the whole X? 18:12 what you need is only for psi(x_i,y_i) to remember y_i and the index i 18:13 no 18:13 psi(x,y)=cos(x)*log(y) 18:13 you will can work directly with the psis 18:14 -!- heiko [~heiko@host86-179-192-248.range86-179.btcentralplus.com] has joined #shogun 18:14 you will never need to know the value of x at no point in training or testing 18:14 -!- romi_ [~mizobe@187.66.121.115] has joined #shogun 18:14 providing that psis are precomputed, or? 18:15 right! 18:15 and our derived psi class takes care of precomputing them in the style which you like 18:15 e.g. precomputing on the fly from x's and y's like you and nico are used to do 18:15 ok, I understand what you mean 18:16 but we can also load psis from disk or an SSD on demand (thats why the getter for required single psi(x_i,y)) 18:16 your point implies that m_features should not be in CStructuredModel, right? 18:17 right! 18:17 because you can still load them if necessary form SSD by get_your_psi 18:17 even when they do not fit into your mem 18:18 that would be scalable ;) 18:18 -!- heiko1 [~heiko@host86-180-43-237.range86-180.btcentralplus.com] has joined #shogun 18:18 and a derived psi class could take care of that loading on demand or whatever 18:18 stop me if I am talking crap 18:18 -!- heiko [~heiko@host86-179-192-248.range86-179.btcentralplus.com] has quit [Ping timeout: 256 seconds] 18:18 may happen ;) 18:18 aham, so m_features could even dissapear from LinearSOMachine 18:18 alexlovesdata: haha ook :D 18:19 right because it asks the getter member to provide the next psi 18:19 yeah! 18:19 alexlovesdata: ok, so I can understand that but you have still to convince with Psi having the Argmax :) 18:20 hehehe 18:20 so the goal is t ocompute argmax_y w*psi(x_i,y) right? 18:21 yes 18:21 I look up something 18:22 -!- puffin444 [472e31fb@gateway/web/freenode/ip.71.46.49.251] has quit [Quit: Page closed] 18:24 so what happens if you have prior knowledge about how to compute the psis from x and y 18:24 encoded in your derived class 18:24 -!- puffin444 [472e31fb@gateway/web/freenode/ip.71.46.49.251] has joined #shogun 18:24 then you can write a very efficient argmax 18:24 as memeber of this class 18:24 which exploits properties of the psi to compute argmax 18:24 to skip some candidate y's 18:25 example y \in \RR^d 18:25 x in \RR^d 18:25 still, I see the relation in the other way; argmax has psi as a member 18:26 shogun: Heiko Strathmann master * rdaeea81 / examples/undocumented/libshogun/statistics.cpp : put different values to examples - http://git.io/r9gYDA 18:26 shogun: Heiko Strathmann master * r724eb3b / examples/undocumented/libshogun/statistics.cpp : Merge pull request #570 from karlnapf/master - http://git.io/CqY94g 18:26 alexlovesdata: would that fit for what you are saying? 18:27 I think it would 18:27 psi(x,y)=cos(sum(x))*y 18:27 psi is one d 18:27 then for w>0 and cos(sum(x))>0 you can skip all negative y's 18:28 the argmax is a function ... 18:28 you want to make it a class? 18:28 yes 18:29 it's already a class in the last version of the code 18:29 we cannot afford to use function pointers 18:29 or better psi(x,y)=cos(sum(x))*y*difficultcomplexbutpositivefunctionof(x,y) 18:29 then an efficient argmax can just look at the signs of the first two terms 18:30 and skip the difficultcomplexbutpositivefunctionof(x,y) 18:30 if psi is one dimensional 18:30 does that serve as an example 18:31 my argument for making psi a member is that it needs only the w and knowledge about the structure of psi and y 18:31 this is already present in the psi class 18:32 ok 18:32 but thinking of argmax as a class too 18:32 which is no contradiction 18:32 because a derived class could call a member argmax, right? 18:32 your idea should fit also if psi IS the member of argmax 18:32 alexlovesdata: yes 18:33 your idea should fit also if psi IS the member of argmax: yes 18:33 I agree 18:33 -!- blackburn1 [~blackburn@188.168.3.9] has joined #shogun 18:33 hmm, C++ has no real pope for it, what a pity 18:33 alexlovesdata: for what? 18:34 for being an instance which tells us what to choose and can never make a mistake  :) 18:35 with the psi being member of the argmax you would call then argmax->psiclass->getnextpsi in optimization 18:36 toget psi(xi,yi) 18:36 also possible ... 18:36 it is because IMHO to have argmax inside psi is not intuitive 18:37 I would be tempted to ask why not intuitive 18:41 but I do not want to go on your nerves 18:41 haha 18:41 no problem 18:41 you can tell nico to berate me fofr today :) 18:41 because Psi doesn't need to know anything about the argmax in order to do its task 18:41 right 18:42 but sometimes the argmax can use specialized information about the psi for computation 18:42 and I think technically it would not make the argmax more special by putting into the psi, right? 18:43 that is still in the direction of argmax needs psi 18:43 but not psi needs argmax :D 18:43 right 18:43 but psi also needs no getter 18:43 but the getter needs psi 18:44 ? 18:44 thats why the getter which delivers the i-th psi is a member 18:44 could be that we are stuck in a question which needs a pope ... :) 18:44 yeah 18:45 or you do as you prefer as long as we can implement a specialized argmax for special psi 18:45 it feels better if we all agree 18:46 I would attach functions to the data classes ... but I will not require that from another ... because that is a style matter 18:46 (a papal matter) 18:46 alexlovesdata: I am curious - can one use latent svm with not only bounding box but some transformations like rotation or perspective? 18:47 I can agree on anything which allows users to implement specialized argmaxes and psi-getters 18:47 I think for general users yes 18:47 thats why I want an abstract psi 18:47 so that any guy who needs something more weird can program it 18:48 yes, that's important too 18:48 so that construction of the actual psi cn be done outside the solver code 18:50 except for where it is unavoidable 18:51 maybe I forgot the important point: 18:52 which I had mentioned just now ... splitting solving the problem from constructing the psi 18:52 because with the current interface that is inside the solver 18:53 I am sorry but I don't understand 18:54 void set_labels(CStructuredLabels* labs);  /** set features * * @param feats features */ void set_features(CFeatures* feats);  /** computes \f$\Psi(\bf{x}, \bf{y}) \f$ */ SGVector< float64_t > compute_joint_feature(int32_t feat_idx, int32_t lab_idx) 18:55 1. now we store the features and labels in memory (can be loaded on the fly ... your virt memory will like that :D ) 18:56 2.  compute joint feature is now inside the solver class, 18:56 with a getter it would be outside the solver algorithm 18:56 with a psi class and a getter it would be outside the solver algorithm 18:57 no one understands me :'-(( 18:57 :) 18:57 I need to check MKL regression .. wasbroken in 0.10.0 and 1.1.0 for matlab with custom kernels 18:58 alexlovesdata: are you the author of MKL in svmlight? 18:59 no 18:59 that was marius kloft 18:59 alexlovesdata: it is not like the compute join feature is inside the solver 18:59 aham 18:59 but I have a little bit insight in it 18:59 we had some issue there 18:59 with LINADD optimizations 18:59 basically it is broken 19:00 ok, that part I never looked into 19:00 alexlovesdata: the idea at that moment was that the model has a in a member the joint feature function and provides this compute_joint_feature for the solver 19:00 ok 19:00 alexlovesdata: since the solver does not have a reference to the psi function directly but a reference to the model 19:01 I understand ... but that requires to store  CStructuredLabels* m_labels;  /** feature vectors */ CFeatures* m_features; 19:01 with an external psi class this would be transparent 19:01 or do some complex hacks which insert artificially a psi 19:02 I am strongly for keeping that computation out of the solver and let the psi getter do that job 19:02 because then you can init the solver with a nando-style psi 19:03 alexlovesdata: would you mind to sketch in a class diagram or using gist how would you like it to be then? 19:03 or an object detection psi 19:03 or a custom psi 19:04 I am mathemtician 19:04 n4nd0: i can do that for ya 19:04 wiking: ok 19:04 I could write an example header, ok?, but I am not familiar with diagrams 19:04 alexlovesdata: ok 19:05 alexlovesdata: afaik i know what you'd like to do here so i'll try to sketch it up in gist 19:05 and let you and n4nd0 check it out 19:05 great! thank you! 19:05 nw 19:05 nw = ?? 19:06 i'll post it on the mailing list 19:06 nw = no worries 19:06 nice 19:06 hi nando, I hope you can live with my blabla ;) 19:08 :D 19:09 n4nd0: I failed with 'curl' word :) 19:10 alexlovesdata: sure no problem! it's good to talk and discuss 19:10 blackburn1: noooooo 19:10 I have absolutely no idea where to put any curl here :D 19:11 alexlovesdata: I did some modifications to a diagram I was using introducing today's conversation 19:21 alexlovesdata: can you take a look to it and tell me if it represents what you said? 19:21 http://dl.dropbox.com/u/11020840/shogun/diagram.pdf 19:21 thank you! 19:21 it is the left-most part 19:21 I take alook 19:22 the arrow means derived class, right? 19:24 the 45 degree rotated cube means class is member of another? 19:25 at the first sight looks nice 19:25 should give us the possibility to do what we need ... 19:25 wiking: what do you think? 19:25 alexlovesdata: just checking 19:26 alexlovesdata: arrow derived class and the cube member yes 19:26 ah yeah 19:26 one thing 19:26 I think it strictly means weak aggregation or something like that, but member is fine :D 19:26 get_psi (lab_idx, feat_idx) 19:27 ok never mind 19:27 it's ok 19:27 i mean essentially it'd be the same idx or? 19:27 probably compute_joint_feature would use get_psi again? 19:27 or can it be that you want get_psi(0,1) ? 19:27 ok yeah you may actually want to do that... so ok 19:28 I think feat_idx refers to the index in x_i 19:28 alexlovesdata: yes, compute_joint_feature is get_psi 19:28 lab_idx refers to the index into all possibly Ys 19:28 not into all possibly Ys 19:28 right? wrong? 19:28 alexlovesdata: well afaik you cannot have all the possible Ys 19:28 alexlovesdata: only the ones that are present ;) 19:29 but the index into the Ys we got in training data 19:29 like the true Ys 19:29 ^ what n4nd0 means here ;) 19:29 youe are right, I agree 19:29 but I think lab_idx would be not the index into y_i , right? 19:29 otherwise I should go to bed soon 19:29 yes, it is 19:29 but don't go to bed :P 19:30 I need to be awake until 1:30 am :( 19:31 why did you say that lab_idx is not the i in y_i? 19:31 so it is the index in y_i  ? 19:31 alexlovesdata: not in but of 19:32 so i guess y_{lab_idx} 19:32 ok 19:32 it actually refers to a given y in the set 19:32 so lab_idx is NOT the training data index, but the index into all available values for Y ? 19:33 that I would think 19:33 I get myself a black tea again 19:33 alexlovesdata: actually it is 19:33 index in the training data index 19:33 yes, in the training data index 19:34 or i understand it as such 19:34 me too 19:34 ok if n4nd0 as well then we are good ;) 19:34 I mean, for me the index into all available values for Y makes no sense 19:34 as it can be an infinite set 19:34 -!- romi_ [~mizobe@187.66.121.115] has quit [Ping timeout: 260 seconds] 19:34 and there's not order defined there 19:35 countable but not finite ;) 19:35 ok then I do not get it currently 19:38 never mind ... temporary confusion 19:39 -!- romi_ [~mizobe@187.66.121.115] has joined #shogun 19:42 nice that we could agree today ... 19:47 hard discussion tonight 19:47 \o/ 19:49 :) 19:49 n4nd0, hey 19:49 gsomix: hi 19:50 can you concretize about director classes that you wish? 19:50 yeah sure, I don't know if you read the conversation about it with sonney2k 19:51 basically is that I think it would be a good idea to have the argmax and the psi function of SO with director classes 19:51 so we can prototype in python and so on 19:51 wiking, alexlovesdata: do you think it would be good to have that? 19:52 yes, sounds practical :) 19:52 second that 19:53 maybe we should prioritize the wishes of directors among all objects ... what people use most?? 19:53 gsomix: what do you think? 19:53 I have no idea how hard or how much time does it take to do these director classes 19:54 gsomix: you have done any already right? 19:54 n4nd0, I just can tell you I'll do whatever you want :) 19:54 n4nd0, e.g. DirectorDistance, last day 19:55 ok 19:57 so let's wait some days so we can say that the design of these classes is better established and I will tell you about it 19:58 n4nd0, ok 19:58 I am going now, talk to you later guys! 19:59 -!- romi_ [~mizobe@187.66.121.115] has quit [Ping timeout: 252 seconds] 19:59 -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Read error: Operation timed out] 20:02 anobody has an idea how the regression label class is initialized in matlab? 20:05 blackburn: what do the scores for the german sign recognition data? 20:08 alexlovesdata: ah I stopped at 97.84% 20:08 not enough time to try different hogs, etc :( 20:09 seems to be +0.4 or so 20:09 ok 20:09 yes colors are great 20:09 for curiosity I tried to put test images to train ones 20:09 99.84% :D 20:09 maybe they did??? 20:10 hmm, the winner got pretty much like 99.84 20:10 winner got 99.46% 20:10 well they use LeNet 20:10 I mean there is this transductive stuff 20:10 yes I just wanted to say I could try transductive 20:11 :D 20:11 but unfortunately I have to get all things done in 4 hours 20:11 with a neural net it could be implicitly transductive 20:11 so I just claim my 97.82 w/o any cheating 20:11 well but for a thesis you can point out that there are some hard cases missing in training and adding them ... blablabla 20:12 haha 20:12 -!- romi_ [~mizobe@187.66.121.115] has joined #shogun 20:13 alexlovesdata: I have other thing that makes me able to claim I've got 100% accuracy :D 20:14 rejects 20:14 also nice ... in particular if you can reject automatically 20:14 alexlovesdata: I just threshold outputs 20:14 with high threshold I get no errors *at all* 20:15 then you could grab similar data from the web, add it to training and get 99.99% 20:15 however 50% are rejected hen 20:15 haha 20:15 thats the ML trick of the day 20:15 :D 20:15 alexlovesdata: too bad I spent too much time on fun instead of training efficient classifier :D 20:19 aren't we all doing it similar? :D 20:20 i.e. I managed to cite Karl Popper but did not do overlapping HoG 20:20 because it was funnier 20:20 and noetherian rings, too? 20:21 no, it is impossible I think 20:21 alexlovesdata: okay lesser cheating - added to trainset images from testset with errors :D 20:26 only 183 images actually 20:26 but then did images from the test set improve which have not been added, as well? 20:26 I am not sure I understand that 20:27 what do you mean? :) 20:28 so you added images from testset to trainset, this implies that all added images will be classified well (because SVMs overfit terribly) 20:31 hmm yes probably 20:31 but were there images which you did not add to the trainset, which have been classified wrongly before and then have been classified correctly after 20:31 i.e. these images would have profited from improved generalization by adding the 183 other 20:32 and not from mere overfitting 20:32 no I added all images there I had errors 20:33 results will be in a min I think 20:33 alexlovesdata: does SVM really overfit terribly? 20:34 yea, usually AUC=100 on training data 20:35 alexlovesdata: what about NNs then? 20:35 I have no idea for neural nets ... 20:37 alexlovesdata: in my world it was thought that NNs overfit and SVMs are better because they do not overfit so much 20:38 alexlovesdata: wow adding 183 images lead to 99.69% 20:38 these 183 images would make me a winner of a contest 20:39 huuh 20:39 on training data with what I work get AUC=100 20:39 error=0 20:39 alexlovesdata: I feel confused because you make me feel all the capacity control, blabla is useless stuff :) 20:40 and probably all that stuff is useless for real 20:40 well capacity control is made for getting a reasonable error on testing data (cross-validation) 20:40 it is not made to tune error rate on train set = error rate on test set 20:41 isn't that for some generalization ability? 20:42 I mean I thought max margin is the point of good generalization 20:42 am I wrong? 20:42 alexlovesdata: I remember you are a big fan of our 'president' - let you become a fan of our parliament http://cs304702.userapi.com/v304702563/1de3/Spom2VHmg4o.jpg 20:47 oops not 183 but 276 20:49 yes, it is for generalization but still you get zero error at trainingdata 21:04 what does the image mean?? - you can explain me tomorrow, I go home, 12 hours is enough! 21:05 alexlovesdata: image? one I send? 21:05 well deputy playing teddy bear 21:06 :D 21:06 -!- alexlovesdata [82955843@gateway/web/freenode/ip.130.149.88.67] has quit [Ping timeout: 245 seconds] 21:09 -!- blackburn [d5578d64@gateway/web/freenode/ip.213.87.141.100] has quit [Ping timeout: 245 seconds] 21:47 -!- puffin444 [472e31fb@gateway/web/freenode/ip.71.46.49.251] has quit [Ping timeout: 245 seconds] 21:49 blackburn1, SVMs don't overfit and your SVM giving that high accuracy on test data certainly does not 22:00 -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun 22:04 -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] 22:19 -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun 22:23 sonney2k, hey 22:24 gsomix, ho :) 22:24 EuclidianDistance vs DirectorDistance 10:37 (in seconds) 22:25 gsomix, how is it going? 22:25 sonney2k, working, programming, building. 22:25 gsomix, you mean again 10 times slower right? 22:25 sounds good 22:25 sonney2k, nope. 3.7 times 22:26 ahh ok - I guess you have the non-optimized atlas version like blackburn1 22:27 gsomix, you could do a director for a general kernel machine 22:28 next I mean 22:28 sonney2k, ok 22:28 same with general linearmachine 22:28 maybe these two should be next 22:29 I think the only important thing to overload here is the train method 22:29 that's about it 22:29 sonney2k: by the way, did you read part of conversation wiking, alexander and I had before? 23:07 -!- wiking_ [~wiking@huwico/staff/wiking] has joined #shogun 23:20 -!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 240 seconds] 23:21 -!- wiking_ is now known as wiking 23:22 n4nd0, superficially yes 23:38 sonney2k: ok, just in case you had a comment or whether psi should be member of argmax or vice versa :D 23:48 sonney2k: hmm do not overfit at all? 23:48 that has the hop topic 23:48 you all confusing me with contradictive claims :D 23:48 good night guys 23:54 ah, btw, google summer of building report http://instagr.am/p/Lf3WXUMs4H/ 23:55 first wall 23:55 hehe 23:55 .___. 23:55 --- Log closed Thu Jun 07 00:00:41 2012