Open in new window / Try shogun cloud
--- Log opened Wed Jun 22 00:00:10 2011
@sonney2kblackburn, again good luck today and nite all00:15
blackburnsonney2k: thanks :)00:15
blackburngood night00:15
-!- f-x [~user@] has quit [Remote host closed the connection]01:24
-!- blackburn [~blackburn@] has quit [Read error: Operation timed out]01:46
-!- in3xes [~in3xes@] has quit [Ping timeout: 263 seconds]08:56
-!- f-x [~user@] has joined #shogun11:24
alesis-noviksonney2k, are you around?11:37
@sonney2kalesis-novik, whats up?12:15
@sonney2kf-x, around?12:16
f-xsonney2k: here12:16
alesis-novikI was wondering if a) the current git version links properly12:16
@sonney2kf-x, did you read my comment about merging with pull --rebase?12:16
f-xsonney2k: yes.. will do that from now.. i thought there was no difference until now..12:17
alesis-novikand b) what's the recommended use of SGVector/SGMatrix as parameters or returned stuff12:17
@sonney2kf-x, yeah everyone had to learn that lesson12:17
@sonney2kf-x, please don't forget  the minor fixes I requested when merging your patch12:18
@sonney2kalesis-novik, compiling to test12:18
@sonney2kalesis-novik, b) just    SGVector<float64_t> foo(); and void foo(SGVector<int32_t> x);12:19
alesis-noviksonney2k, I get that part, but what about memory allocation and shallow copy vs deep copy12:20
@sonney2kalesis-novik, never copy if possible12:20
@sonney2kand allocation with new[]12:21
@sonney2kalesis-novik, it compiles...12:21
alesis-novikSo the idea is to rely on the user not to break stuff12:21
@sonney2kmaybe you  need to run git clean -dfx12:21
@sonney2kyes. there is no alternative when one does not want to copy all the time12:21
@sonney2kin the end it is like using java and references all the time12:22
alesis-noviksonney2k, ok, I'll get my code to do that. Ideally I'll also use SGVector and SGMatrix for fields as well, seems a lot less messy12:22
f-xsonney2k: In StreamingStringFeatures, there will only be one string right?12:24
f-xit seems many functions from StringFeatures won't apply12:24
f-xlike get_transponse etc12:25
f-xand what about the subsets?12:26
-!- blackburn [~blackburn@] has joined #shogun12:29
blackburnit's f-ing over!12:29
alesis-novikCongratulations blackburn12:30
alesis-noviksonney2k, yep, that fixed it12:30
blackburnalesis-novik: thanks!12:30
blackburnsonney2k: are we interested in R? ;)12:40
blackburnI know two guys - one interested in R, one interested in common lisp :D12:57
@sonney2kblackburn, congrats!13:16
@sonney2kblackburn, yes we have R support :)13:16
blackburnsonney2k: thanks :)13:16
bettybooha ha13:16
@sonney2kblackburn, hope you passed with some good mark13:16
blackburnsonney2k: 4/513:16
@sonney2kblackburn, as you intended13:16
blackburnwell it wasn't really fair :) it could be 5/513:17
blackburnthe problem is - I know a little *real* math like I do with dim.reduction now and I simply can't 'rote learn' these useless things :D13:18
@sonney2kblackburn, ahh doesn't matter - what matters is that you passed good enough and that you have now spare time13:18
@sonney2kf-x, no subsets just the string as is13:19
blackburnsonney2k: and in fact I'm not going to have 'red diploma' - 'summa cum laude' analogue - so 4 is good for me too :D13:20
blackburnthis week I will finish LMDS!13:21
-!- f-x [~user@] has quit [Ping timeout: 260 seconds]13:23
blackburnsonney2k: I think I should revise my plan for mid-term13:24
-!- sploving [~root@] has joined #shogun13:33
alesis-novikAnyone know if the SGVector and SGMatrix things are working with octave_modular?13:41
blackburnI don't13:42
-!- in3xes [~in3xes@] has joined #shogun13:44
-!- f-x [~user@] has joined #shogun14:06
splovinghello blackburn14:13
-!- in3xes [~in3xes@] has quit [Quit: Ooops..!!]14:23
alesis-novikBah, everything was working, then I upgraded to use SGVector and SGMatrix and now I'm getting Seg faults... This is frustrating14:39
-!- heiko [] has joined #shogun14:51
-!- sploving [~root@] has left #shogun []15:15
-!- in3xes [~in3xes@] has joined #shogun15:24
-!- f-x [~user@] has quit [Ping timeout: 260 seconds]15:35
-!- f-x [~user@] has joined #shogun15:41
-!- f-x is now known as Guest9083915:41
-!- Guest90839 [~user@] has quit [Ping timeout: 260 seconds]15:47
alesis-novikWeird. When testing sampling from Gaussian in c++ it seems to work fine. In octave it gives weird values(with stuff like e-158)15:48
-!- f-x` [~user@] has joined #shogun15:55
-!- heiko [] has quit [Ping timeout: 258 seconds]16:12
-!- heiko [] has joined #shogun16:25
-!- in3xes_ [~in3xes@] has joined #shogun16:38
-!- in3xes [~in3xes@] has quit [Ping timeout: 240 seconds]16:40
-!- in3xes_ is now known as in3xes16:40
-!- heiko [] has quit [Ping timeout: 258 seconds]17:02
-!- f-x` [~user@] has quit [Remote host closed the connection]17:53
-!- heiko [] has joined #shogun17:58
-!- heiko [] has quit [Ping timeout: 258 seconds]18:47
alesis-noviksonney2k, are you around?19:23
alesis-novikwhen you read this, could you tell me if its better to try not to copy to SGVector/Matrix or be consistent.19:26
alesis-novikIn Gaussian in get_mean I could just assign it, but in get_cov I reconstruct the covariance, thus it falls on the user to take care of memory19:27
alesis-novikso is it better to use different things or just copy in both?19:27
-!- heiko [] has joined #shogun19:28
alesis-novikAlso, I think I've build up  a nice big patch. Should I just submit a pull request now, so I can start doing small incremental changes from now on?19:28
@sonney2kalesis-novik, try to avoid copying whenever reasonably possilbe20:21
@sonney2kif not possible mark it in the documentation20:21
@sonney2k*and* set the do_free flag to true20:21
@sonney2kalesis-novik, and yes send a pull request20:29
@sonney2kalesis-novik, what about octave-modular - use git blame and give me bug reports don't wine alone in the dark ;-)20:29
CIA-32shogun: Baozeng Ding master * rd75d754 / examples/undocumented/java_modular/ : change to accept an optional argument (.java file to check) -
alesis-noviksonney2k, I was wondering if there's a specific reason why 1-by-1 matrices are not valid for SGMatrix when using python-modular?20:48
@sonney2kalesis-novik, with 1-by-1 you mean a matrix with 1 col / 1 row?20:49
alesis-noviksonney2k, yes20:50
@sonney2kalesis-novik, then you must be doing sth. wrong20:50
@sonney2kthey are valid20:50
alesis-novikOk, I will try that. Ah, I might have forgotten to add another set of []. Silly me.20:51
@sonney2kthat explains it20:52
@sonney2kthe other thing is a vector20:52
alesis-noviksonney2k, do you by any chance remember the octave equivalents for the SGVector and SGMatrix?20:54
@sonney2ksame thing20:54
@sonney2kor do yo mean how to write it in octave?20:54
@sonney2k[1,2,3] (vector)20:54
@sonney2k[1,2,3;4,5,6] (matrix20:54
alesis-noviksonney2k, thanks. I think there is still something wrong with octave, but I'll check and submit any possible bugs after the pull request (so there's actually code to reproduce what I have done)20:57
@sonney2kalesis-novik, I have a clear idea what could have happened with octave.20:57
@sonney2ksince we now don't always *copy* it might be that we overwrite sth20:58
@sonney2kerr free memeory that shouldn't be20:58
alesis-novikI still need to clean up the code after the SGVector and SGMatrix changes20:58
-!- heiko [] has quit [Quit: Leaving.]21:00
alesis-novikHmm, just running init_shogun then quit produces seg fault in octave21:00
@sonney2kalesis-novik, that is `normal'21:01
@sonney2kbug in octave unfortunately21:01
@sonney2kthis bug can only be fixed in octave 3.4 and when someone fixes swig's octave bindings for that version21:02
blackburnnormal segfault *lol*21:02
@sonney2kbasically loading multiple extensions in octave that depend on each other somehow makes octave crash on exit21:02
alesis-noviksonney2k, one more question about SGMatrix :) If I use set_cov(SGMatrix<float64_t> cov), but don't actually store the cov value, should I set the do_free flag on exit?21:02
bettybooalesis-novik, lolomat21:02
@sonney2kbut apart from that it will work21:02
@sonney2kalesis-novik, what do you do with cov when not storing it internally?21:04
alesis-noviksonney2k, I'm decomposing it and storing eigenvectors and eigenvalues21:04
@sonney2kblackburn, any idea? I didn't think of this case though alesis-novik's proposal of setting do_free makes sense21:05
blackburnsonney2k: don't understand, where it could be set?21:05
blackburninside set_cov()?21:05
blackburnbut what if we don't want to free it21:06
blackburnI mean we have some matrix and passing it to that method21:07
@sonney2kok and then?21:07
blackburnwhat if we don't want to free that matrix?21:07
@sonney2kbtw if he sets do_free it won't work21:07
@sonney2kit is a local variable21:07
blackburnah yes21:08
blackburndon't know for sure..21:09
@sonney2kalesis-novik, for now just document this in set_cov - we have to think about ways to resolve this for the general case21:09
alesis-noviksonney2k, just note it in the description?21:10
@sonney2kblackburn, it is actually only a problem for swig related functions- because these will always pass *copies* of the objects21:10
@sonney2kalesis-novik, yes21:10
@sonney2kbaby crying... will be back later.21:11
alesis-novikblackburn, so, how are you feeling with all the exams done?21:12
blackburnalesis-novik: very nice :) btw did you traveled home after exams?21:13
alesis-novikI did last week. Found out the exam results while there. Had to travel around Lithuania as well (to visit everyone I wanted)21:14
blackburnare you in uk now?21:15
alesis-novikblackburn, yeah, back in Edinburgh21:16
-!- f-x [~user@] has joined #shogun21:16
blackburnalesis-novik: have you any studies now?21:17
alesis-novikThe rent contract is until September and I need to take care of stuff to do with my coming PhD (get a new place, open bank account, etc.)21:17
blackburnoh so you graduated already?21:17
@sonney2klooks like the baby gave up uhmmm...21:18
@sonney2kf-x, would you mind submitting the small patch now?21:18
alesis-novikblackburn, I'm done with my exams, but not exactly graduated yet21:18
f-xsonney2k: ok21:19
alesis-novikOh boy, this will get confusing fast... The constructor for Gaussian takes both mean and cov. We can just point the m_mean field where we need, but we will still have to decompose the covariance.21:19
@sonney2kalesis-novik, hmmhh then be consistent and delete[] it?21:22
@sonney2kat least that is what all the label, features, ... objects do: you set_* and then on destruction of the objects the set objects are destroyed21:23
alesis-novikSo decompose cov, copy mean and delete everything on destruction?21:24
@sonney2kthe only other alternative I can think of is a CSGMatrix (or so) that is derived from SGObject and does proper reference counting21:25
@sonney2kalesis-novik, decompose cov, delete[] cov21:25
@sonney2kstore the mean ptr and delete[] mean on destruction21:25
@sonney2k(no copy of mean vec)21:25
CIA-32shogun: Shashwat Lal Das master * r5367d36 / src/libshogun/features/StreamingFeatures.h : Changed return type of get_next_example to bool in StreamingFeatures. -
alesis-noviksonney2k, ok, but I think the idea of a Vector and Matrix derived from SGObject would make things a lot clearer (though quite a bit would need to be rewritten and having no experience I have no idea how well would that work with swig)21:27
blackburnthese SGtypes issues hurts :D21:33
@sonney2kblackburn, well it is the no copy / copy issue21:39
@sonney2kalesis-novik, the problem is that it comes with a huge overhead21:39
@sonney2kalesis-novik, if you used SimpleFeatures for cov you wouldn't have this problem21:39
@sonney2kthough it is an unecessary overhead ...21:40
blackburnsonney2k: do you know some physics machine learning dataset?21:47
alesis-novikStill, this is nothing compared to the software they use at CERN. That thing is horrible.21:47
@sonney2kalesis-novik, please use the delete[] approach. we really everywhere assume that the set_* ptr is delete[] on object destruction so it makes sense to immediately delete it if not needed.21:49
blackburnsonney2k: nevermind, another question: what is the license should be for dataset in shogun-data?21:49
@sonney2ksonney2k, spring glass?21:49
@sonney2kblackburn, great question some CC-* one I guess21:50
blackburnsonney2k: I suggested one student with physics major to find some physics-machine-learning task and add it to shogun21:50
blackburnin applications or examples21:51
@sonney2kblackburn, good examples are always welcome21:52
blackburnsonney2k: if you will have ideas about physics (I know it is crazy a little) - tell me21:52
alesis-noviksonney2k, should I use free_vector()/free_matrix()? That seems to clean up nicely21:53
@sonney2kblackburn, what do you think? if we would require that do_free is properly set - we always know when we 'own' the matrix22:02
@sonney2kblackburn, however when doing return some_sgmatrix we would need to clear that flag22:03
blackburnshould we always call free_* for that?22:03
alesis-novikCurrently, I did as you suggested sonney2k: delete via free_matrix and set flat to true on returning copied values22:03
@sonney2kblackburn, yeah in destructor we would then always put a free_* in there instead of delete[]22:03
blackburnlooks good to me22:04
@sonney2kalesis-novik, keep it like this for now - the other thing requires quite a big overhaul we cannot do just now22:04
alesis-noviksonney2k, well, it does work properly (at least in c++), so I see no problem22:05
blackburnbut we should be carefully whenever do_free or not, right?22:05
@sonney2kthis all could actually be sufficient22:05
@sonney2kblackburn, yeah. I mean if someone does set_feature_matrix(mat) with mat not having do_free set the caller has to clean up22:06
@sonney2kif he sets the thing we clean up22:06
@sonney2kif we return the matrix we have to unset the do_free flag22:06
blackburnI see22:07
@sonney2kif we return a copy we have to set the do_free flag and copy22:07
@sonney2k(copy or new object that is)22:07
@sonney2kso that seems to cover all cases22:07
blackburnlooks like22:07
@sonney2kalesis-novik, what do you think? could this work?22:08
alesis-noviksonney2k, sounds good22:10
alesis-novikby unset the flat you mean set it to false?22:11
alesis-novikForgive my inexperience with git, but does rebase destroy the order of commits and thus should always be force pushed to the branch?22:33
blackburnnot it won't destroy commits22:34
alesis-noviknot the commits themselves, but the order22:36
blackburnwhy it should?22:38
alesis-novikbecause first applies the commits to the merged branch and applies your commits on top of that22:41
blackburnI don't know :(22:48
@sonney2kalesis-novik, yeah you shouldn't push this as it indeed changes the order of commits23:10
@sonney2kthe way I use it is have 2 branches (master aka upstream) and one devel branch23:11
@sonney2kand then I pull from upstream into master and then checkout devel and rebase devel with master23:11
f-xsonney2k: how should a function like obtain_by_sliding_window() for a streaming string class behave? should it return the new features as an SGString* variable?23:13
@sonney2kf-x, don't do more for strings except getting the raw sting23:14
@sonney2kno fancy stuff23:14
@sonney2kjust the string you read23:14
f-xsonney2k: ok.. so basically just the reading part differs23:15
f-xfrom SimpleFeatures23:15
@sonney2kf-x, it is basically simplefeatures without the atod parsing23:15
f-xsonney2k: right.. so all the bit representation stuff is 'fancy' for now?23:15
@sonney2kf-x, well you will need alphabet and potentially a remapping from ascii -> binary23:17
f-xsonney2k: and a check during reading whether each char belongs to the alphabet?23:18
@sonney2kf-x, I am afraid yes (alphabet provides a function for that) - I would use alphabet to create a histogram and then check if that is invalid23:19
f-xokay.. i'll see what functions from stringfeatures i need to retain..23:20
f-xso after each line read, call add_string_to_histogram(), and then check_alphabet()?23:25
CIA-32shogun: Sergey Lisitsyn master * rb38b907 / (4 files): ST* transition for Distribution base class and Histogram -
@sonney2kf-x, that makes at least sense for strings > 256 chars or so in length. belwo that checking each char manually is probably faster23:35
f-xsonney2k: and we have something like SGString<ST> fv as a member? or should ST* be used? (like it is done for single_string now)23:39
f-xi mean is there a reason for the different datatypes for features and single_string in StringFeatures?23:40
@sonney2kf-x, I don't understand - just return a SGString<ST> - same type as you expect the inputs in (char,int,float...)23:41
f-xsonney2k: alright.. didn't understand why single_string was of ST* type in StringFeatures.. never mind23:42
@sonney2kignore the single string and fancy stuff23:43
blackburnsonney2k: how to delete current line in vim? sorry for lame question :)23:46
@sonney2kblackburn, dd23:46
blackburnsonney2k: ok how to undo23:46
blackburnsonney2k: vim is wonderful!23:47
@sonney2ktry gf when the cursor sits on a file name23:48
@sonney2kor try k when you are on e.g printf23:48
blackburngf is not working :(23:49
blackburnsays can't find file distributions/Distribution.h in path23:49
blackburnI guess it should be set up somehow23:49
@sonney2kblackburn, you need to be in libshogun for that23:49
@sonney2kin that path23:50
blackburni see23:50
CIA-32shogun: Sergey Lisitsyn master * r1c13dc4 / (2 files in 2 dirs): ST* transition for PPWM -
@sonney2kblackburn, or do :set path=libshogun;.;/usr/include or so23:55
blackburnsonney2k: will do tomorrow23:55
blackburnsonney2k: distributions are working23:55
blackburnexcept LinearHMM23:55
blackburnwill fix tomorrow too23:55
@sonney2kblackburn, I am still working on sparse features...23:56
blackburnsonney2k: oh.. how much code you have to fix here?!23:56
@sonney2kblackburn, typemaps won't work for some reason23:56
@sonney2knot being applied23:57
blackburnI see23:57
blackburnokay, will go to bed now, see you23:57
blackburnsonney2k: nite23:57
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]23:57
@sonney2kblackburn, bye23:58
--- Log closed Thu Jun 23 00:00:13 2011