Open in new window / Try shogun cloud
--- Log opened Fri Apr 27 00:00:19 2012
-!- n4nd0 [~nando@188.Red-2-137-0.dynamicIP.rima-tde.net] has quit [Quit: leaving]00:34
-!- av3ngr [av3ngr@nat/redhat/x-fdgjejyzjtxwmlhd] has joined #shogun02:30
-!- vikram360 [~vikram360@117.192.187.42] has quit [Ping timeout: 260 seconds]03:01
-!- PhilTillet [~Philippe@vir78-1-82-232-38-145.fbx.proxad.net] has quit [Ping timeout: 260 seconds]04:17
-!- vikram360 [~vikram360@117.192.172.17] has joined #shogun06:10
-!- gsomix [~gsomix@188.168.5.45] has quit [Read error: Operation timed out]08:37
CIA-64shogun: Soeren Sonnenburg master * r4574712 / (3 files in 2 dirs):08:43
CIA-64shogun: don't do SG_FREE on varray.begin08:43
CIA-64shogun: This is now handled in the destructor of v_array and additional free's08:43
CIA-64shogun: only cause double free bugs. - http://git.io/r81grA08:43
shogun-buildbotbuild #739 of cmdline_static is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cmdline_static/builds/73909:05
-!- vikram360 [~vikram360@117.192.172.17] has quit [Ping timeout: 250 seconds]09:10
-!- magicfly [c07c1afb@gateway/web/freenode/ip.192.124.26.251] has quit [Quit: Page closed]09:21
-!- av3ngr [av3ngr@nat/redhat/x-fdgjejyzjtxwmlhd] has quit [Remote host closed the connection]09:32
-!- av3ngr [~av3ngr@60-241-222-244.static.tpgi.com.au] has joined #shogun09:55
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]10:40
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun10:41
-!- av3ngr [~av3ngr@60-241-222-244.static.tpgi.com.au] has quit [Quit: That's all folks!]11:12
sonne|workwiking: looks like we are the lone rangers today...11:15
wikinghehehe people went on holidays or what? :)11:15
sonne|worklooks like11:16
sonne|workI would have wanted to talk to pluskid again about these new sgvector developments but hey11:16
wikingwhat's the final decision on that?11:16
sonne|workthere is no final decision11:16
sonne|workI start to agree with blackburn, i.e., have a very simple rudimentary SGVector that is only used to pass around ptrs (not for long term usage)11:17
sonne|workand some other class that has more overhead, like GC11:18
sonne|workmaybe derive one from the other11:18
sonne|workand the SGIVector (I == interface)11:18
wikingmmm11:18
wikingso basically you are ending up with the shared_pointer concept11:18
sonne|workis then only stuff to be used from the outside for long term11:19
sonne|workno11:19
wikingi mean if you are doing interfacing of SGVector11:19
wikingand such11:19
wikingyou'll have the same thing going on11:19
sonne|workwhat I mean is we have just double* and int for sgvector11:19
sonne|workno more11:19
sonne|workno overhead11:19
wikingyep11:19
wikingso no reference counter there i guess11:20
sonne|workwe use this whenever possible11:20
sonne|workbut for internal class storage of (bigger) sgvectors we use sth like SGIVector11:20
sonne|workthat can be shared and passed around11:20
sonne|workwith reference counts and stuff11:20
wikingyeah11:21
wikingwell that is exactly what i meant about SGIVector will mimic the shared_ptr concept of boost11:21
sonne|workyes true11:21
sonne|workmore or less :)11:21
wikinghehe ok11:21
wikingsounds reasonable11:21
wikingfor11:21
wikingme11:21
wiking:)11:21
wikingdamn too many enters11:22
wiking(doing a ubuntu lts upgrade in the background)11:22
sonne|workthis needs to be discussed with pluskid, blackburn and n4nd0 ...11:22
sonne|workno idea whether they all attempted to upgrade ubuntu yesterday11:22
sonne|workand now are gone :P11:22
wikinghahahahah yeah11:22
wikingmaybe they've all done that and now trying to fix it... i'm just doing it on a server so worst case it's going to crash :P11:23
wikingif that works out11:23
wikingthen i should the same with all my cluster nodes ;P that'll be fun for sure :)11:23
wikingbtw do u know any industrial application of shogun?11:24
sonne|workrumor has it that it is used11:25
wikingoh cool11:25
wikingi wonder what environment11:25
sonne|workno idea11:25
wikingi'd be interested testing shogun's performance on some embedded device11:26
wikingwhether it's feasable11:26
wikingat least small parts of it11:26
sonne|workheh, we ran it on gunnars iphone a while ago :D11:26
wikinghow was it? draining battery? :)11:26
sonne|workjust for fun11:26
wikingi might be able to try on a raspberry pi when i finally receive it :)11:26
sonne|workno applications...11:26
sonne|workcertainly all the linear methods are fast enough for embedded11:27
wikingyes11:27
wikingthat's what i've thought11:27
wikingmaybe we should create a build file for andro and iOS :)11:27
wikingjust for the fun of it11:27
wikingit could even be11:27
wikingthat it only contains only the 'classification' parts11:28
wikingthat it can load up the model11:28
wikingand then just classify it11:28
wikingin case of classifiers... since learning u can do in a cloud as well :)11:28
wikingand then just download the learned model11:29
sonne|workI would say just libshogun11:29
sonne|workand the static cmdline interface11:29
wikingdamn the shipping date is still 15th of august for my raspberry pi11:30
wikinganyhow once i have it i'll give it a go and do some benchmarking ;)11:34
wikingshould be fun11:34
sonne|worktoys are supposed to be fun :D11:36
wikingheheh my wife just won an award :)11:36
sonne|workgood idea to spend it on toys ;-)11:38
sonne|workwhich one?11:39
sonne|workbtw11:39
wikingah it's a local artist award from our hometown11:39
wikingi think there's no money with it only 'fame' :)11:40
sonne|workheh, that reminds me we still need a cool logo :)11:40
wikingahahhahaha11:40
sonne|workwiking: btw are you still in touch with alex?11:44
sonne|workwiking: ahh and do you remember what I wanted to discuss with vojtech? I totally forgot to talk to him11:44
sonne|workahh11:45
sonne|workfood11:45
wikingsonne|work: yeah i'm in touch with him, I'm just finishing now a big chunk of the latent svm code, so when it's committed i'll have another big discussion with him about how to go further... and whether or not he is satisfied with the outcome11:47
wikingsonne|work: mmm vojtech? YES i know! about adding stuff to libqp11:48
wikinglike pr_loqo11:48
wikingsonne|work: so yeah get hold of him and talk about that (pr_loqo and i think it was tron? to libqp)11:50
-!- PhilTillet [~Philippe@vir78-1-82-232-38-145.fbx.proxad.net] has joined #shogun12:33
wikingPhilTillet: hey! r u still planning to do the patches for opencl in shogun?12:33
PhilTillethello wiking :p12:34
wikingPhilTillet: or it should be picked up by somebody else12:34
PhilTilletabsolutely, i've spent the last 4 days reading litterature about svm12:34
wiking?12:34
wikingyou wanna opencl svm12:34
PhilTilletyes12:34
wikingi must say that you'll have problems with that12:34
wikinga lot of people tried various ways to extend svm parallel12:34
PhilTilletwhy? there has been a lot of paper succeeding!12:35
wikingand there are some ways of course12:35
wikingbut neither is a silver bullet12:35
PhilTilleti've already done a prototype on classification (not learning) , and the improvement was something like seven-fold or eight-fold :p12:36
wikingyeah learning is a hard part12:36
PhilTilletbut I have some paper who parallelized smo with success12:36
wikingyeah smo12:36
PhilTilletimproving performances by 20x :p12:36
wikingyou have one from google as well12:36
wikingpsvm12:36
wikingit's using mpi12:36
PhilTilletwell mpi and opencl are completely different :p12:37
wikingyeah i know12:37
PhilTilletcuSVM would be more of an example ^^12:37
PhilTilletanyway, I need to go to eat :p12:37
PhilTilletcya12:37
wiking:(12:37
wikingshit since i just wanted to say12:37
wikingsomething12:37
wikingbut let me know when u r back12:37
PhilTilletwiking, i'm back12:56
PhilTillet:p12:56
wikingok12:56
wikingjust a sec i've fucked up something on my server12:56
wiking:)))12:56
PhilTilletgood job! :D12:56
wikinghahah fixed12:58
PhilTillet:p12:58
wikingi wonder if this works now12:58
wikingbrb12:58
wikingoh no i cannot test it yet12:58
wikingso anyhow have u taken a look at libqp?12:58
wikingif that could be opencl-ized12:59
PhilTillethmmm not at all :p12:59
-!- karlnapf [~heiko@host86-182-161-144.range86-182.btcentralplus.com] has joined #shogun12:59
wikingthat'd be great though13:00
PhilTilletactually i'm not very familiar yet with constrained optimization (that's why i need to read a bit of litterature :D)13:00
* wiking is just testing mosh13:00
PhilTilletI just know that SMO has been successfully cuda-ized :p13:01
wikingThe implemented solver (libqp_splx.c) is a generalization of the method proposed in [1, 2]. It is based on the Sequential Minimal Optimization (SMO) algorithm with an improved working set selection strategy.13:01
PhilTilletwell then I guess yess :p13:01
PhilTilletdoes Shogun relies on libqp for training?13:02
wikingwell more and more things will13:02
wikingbut like svmocas is one example13:03
PhilTillethmmm13:03
wikingand most probably 2 gsoc project this year13:04
wikingis going to use libqp13:04
PhilTilleti see :p13:04
wikingeven 313:04
wiking;)13:04
PhilTilletthen I should more focus on a generic implementation of libqp that could be re-used for svm?13:05
wikingwell check this13:05
wikingsrc/shogun/lib/external/libqp_splx.cpp13:05
wiking     410 src/shogun/lib/external/libqp_splx.cpp13:05
wiking410 lines13:05
wikingso it's not that big13:05
PhilTilletwell indeed it's not that big :p13:10
sonne|workPhilTillet: that is not the bottleneck13:11
sonne|workthe bottleneck is the thing I gave you :)13:11
PhilTilletwhat do you mean? :p13:12
-!- wiking_ [~wiking@78-23-189-112.access.telenet.be] has joined #shogun13:13
-!- wiking_ [~wiking@78-23-189-112.access.telenet.be] has quit [Changing host]13:13
-!- wiking_ [~wiking@huwico/staff/wiking] has joined #shogun13:13
wiking_woah this mosh works great :)13:13
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 252 seconds]13:15
-!- wiking_ is now known as wiking13:15
PhilTilletsonne|work : I have thought about a cleaner way to implement OpenCL without #ifdef everywhere :p making the functions not #ifdef'ed in the .h, and make another .cpp to implement them, and if USE_OPENCL is not defined, just return an error message like "No OpenCL support"13:18
sonne|workPhilTillet: this \sum_i alpha_i K(x,x_i) is the operation that kills time13:19
sonne|workall the rest is almost insignificant13:19
PhilTillettraining is insignifiant? :o13:19
sonne|workthis operation is what takes most of the time in training13:19
-!- karlnapf [~heiko@host86-182-161-144.range86-182.btcentralplus.com] has quit [Quit: Leaving.]13:19
PhilTilletoh, yes13:20
PhilTilletwell I'll keep on reading more and try to come up with an implementation :p13:20
PhilTilletbut a lot of guys have succeeded so i'm sure that it's possible :p13:22
-!- Priyans [~Priyans@115.248.130.148] has joined #shogun13:24
PhilTilletIt will just take some time :D13:27
-!- wiking_ [~wiking@78-23-189-112.access.telenet.be] has joined #shogun13:32
-!- wiking_ [~wiking@78-23-189-112.access.telenet.be] has quit [Changing host]13:32
-!- wiking_ [~wiking@huwico/staff/wiking] has joined #shogun13:32
-!- wiking [~wiking@huwico/staff/wiking] has quit [Read error: Connection reset by peer]13:32
-!- wiking_ is now known as wiking13:32
-!- PhilTillet [~Philippe@vir78-1-82-232-38-145.fbx.proxad.net] has quit [Remote host closed the connection]13:59
--- Log closed Fri Apr 27 14:18:08 2012
--- Log opened Fri Apr 27 14:28:58 2012
-!- shogun-toolbox [~shogun@7nn.de] has joined #shogun14:28
-!- Irssi: #shogun: Total of 9 nicks [1 ops, 0 halfops, 0 voices, 8 normal]14:28
-!- Irssi: Join to #shogun was synced in 6 secs14:28
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]14:29
-!- wiking [~wiking@vpna052.ugent.be] has joined #shogun14:29
-!- wiking [~wiking@vpna052.ugent.be] has quit [Changing host]14:29
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun14:29
-!- blackburn1 [~qdrgsm@188.168.2.179] has joined #shogun14:42
-!- Priyans [~Priyans@115.248.130.148] has quit [Quit: Leaving]15:43
-!- blackburn1 [~qdrgsm@188.168.2.179] has quit [Ping timeout: 260 seconds]15:57
-!- henri [2e1fd566@gateway/web/freenode/ip.46.31.213.102] has joined #shogun16:24
henrihi there16:24
henriI have a problem of memory desallocation16:25
henriI use opencv for computer vision apps, and I initialize shogun matrix from cv::Mat object like this :16:25
henricv::Mat cvmat(100,100,CV_32FC1,cv::Scalar(0.0));16:25
henriSGMatrix<float> sgmatrix((float*)cvmat.data, cvmat.rows, cvmat.cols);16:25
henriproblem is when for example I do SG_UNREF(svm), then I got *** glibc detected *** /home/(...): double free or corruption (out): 0x0000000000a2f1c0 ***16:26
sonne|workhenri: where do you put this matrix?16:29
sonne|workinto SimpleFeatures<float> ?16:29
henriyes16:29
sonne|workhenri: argh yes that explains it16:29
henrithis could be written like that : CSimpleFeatures<float>* features =new CSimpleFeatures<float>((float*)samples.data,samples.rows,samples.cols);16:29
sonne|workwe currently assume to 'own' the memory16:29
henrisamples is cv::Mat struture16:29
sonne|workso when the simple feature object is destroyed it will SG_FREE the matrix16:30
sonne|workand I guess opencv does the same16:30
henriindeed16:30
sonne|workwiking, blackburn -> that is why we need the new sgvector stuff ...16:30
sonne|workhenri: we have some plans to avoid copying but that hasn't been done yet16:31
sonne|workhenri: so for now you will have to feed a copy of the opencv matrix into shogun16:32
sonne|workgsomix: are you around?16:32
wikingheheheh :)16:32
henriI don't understand why I get this error from shogun because opencv free the matrix after shogun16:33
henriI tried to add ref to the matrix but same error16:33
henrisonne|work: Wekk ok, should I go this way ? :: SGMatrix<float> sgmatrix = new SGMatrix<float>((float*)cvmat.data, cvmat.rows, cvmat.cols);16:35
wikingmm16:35
henri+ *16:35
wikinghenri: u should copy the cvmat.data16:35
henriwiking: ok tx16:35
wikingotherwise you'll end up freeing twice the float array16:35
wikinghenri: can we ask u what's your application?16:36
henrisure, I try to recognize video events16:37
wikingah cool16:37
wikingwhich svm are u using in shogun?16:37
henrifor now I used linear kernel on bow features16:39
henribut its just a beginning16:39
wikingand how is it working out for you?16:39
wikingah ok16:39
wikingif u use BOW and linear kernel16:39
wikingi might suggest to test HomogeneousKernelMap preprocessor on your features16:39
wikingyou'll have better classification results16:40
henriwhat is it ?16:40
henriMy main goal is to keep time information which are lost in BOW features16:41
wikingit's based on this paper: http://www.robots.ox.ac.uk/~vgg/publications/html/vedaldi10-abstract.html16:41
henriI am gonna use strings kernels or kind of fisher kernel to do that16:41
wikingit basically allows you to have similar accuracy when u use histogram intersection kernel or jensen-shannon kernel on your histogram features but using linear svm16:42
wikingit's pretty cool16:42
henriproblem is that strings kernel in shogun handle only a short alphabet16:42
henriI will provide advancement information if you want to16:43
wikingyeah sure16:43
wikingsince i use shogun for image processing as well16:43
henriAre you one of the shogun developpers ,16:43
henri?16:43
wikingwell i'm doing now gsoc for shogun16:44
wikingbut for example this homogeneous kernel mapping trick has been ported by me to shogun16:44
henrion what part ?16:44
wikingas i needed linear svm but with the similar accuracy as if i'd use jensenshannon kernel16:44
wikingi'm implementing the latent svm module for shogun16:45
wikinghenri: u use this as your research for uni?16:45
henriyes16:46
henriAnd I would help shogun with pleasure16:47
henriIf I can16:47
wikingcool! anyhow if u have a data set where u wanna apply latent svm let me know since i'm just trying to build several scenarios where it could be applied. i.e. if u not only want to label an even in the video but you wanna give e.g. a bounding box for where that event happened that could be done via latent svm16:48
henriI can try, what data base have you already processed ? Thans for the article I am reading it !16:53
henri=)16:53
wikingwell currently i'm testing with mammal identification in an image16:54
henriwiking: allright, I have to go now, I will be back next week.17:00
wikingalright!17:00
wikingcya17:01
henriwiking: the name is not henri but I will try to make you remember me, even with a different name ^^ cya17:01
wikinghehehe ok :)17:01
-!- henri [2e1fd566@gateway/web/freenode/ip.46.31.213.102] has quit [Quit: Page closed]17:02
gsomixsonney2k, sonne|work hey17:25
gsomixsonney2k, sonne|work just write me. I'm ready to go. :)17:33
sonne|workgsomix: do you see blackburn today?17:33
sonne|workI am thinking simplifying SGVector to only be double* and int17:34
sonne|worknot refcount or anything17:34
sonne|workand then have a derived SGIVector (I == interface)17:34
sonne|workwhich we use only for returning things to the outside which keeps proper refernce counts and everything17:34
sonne|workso we have zero overhead when using SGVector17:35
sonne|workand can afford quite some for SGIVector17:35
gsomixsonne|work, I saw him at the uni, but he escaped.17:35
sonne|workgsomix: while wiking likes this I would prefer to discuss this with blackburn, n4nd0, pluskid17:36
sonne|workbut that would be the plan17:36
wikingmailing list?17:36
sonne|workgsomix: maybe you can start to create some draft implementation17:36
wikingit might be more efficient to discuss it there17:36
sonne|worktrue17:36
sonne|workgsomix: e.g. create an SGIVector which has refcounts in the same way like SGObject but with some boolean that it refcounts for this thing are not needed)17:37
wikingoh fucking hell17:37
sonne|workand simplify the sgvector class17:37
sonne|workwiking: ?!17:37
wikingmay 8 ... kdd deadline! :(17:37
wikinganother day of paperworks for me it seems:D17:38
sonne|workwork wiking work!17:38
wikingwell it's just gonna be copy paste first17:38
wikingand then add some extra graphs17:38
wikingas i'm sure my last submission is going to be refused :D17:38
wikingbut i'll get the notification tomorrow ;)17:44
gsomixsonne|work, we plan to use SGIVector as pointer, right?17:55
-!- blackburn1 [~qdrgsm@188.168.13.62] has joined #shogun17:58
blackburn1sonne|work: sonney2k bounce18:01
blackburn1sonney2k: I'd like to mimic or use valarray in the lightweight SGVector18:03
sonne|workgsomix: no18:04
sonne|workgsomix: the same way as before18:04
sonne|workblackburn1: new day new idea:18:05
blackburn1sonne|work: no old idea18:05
blackburn1wiking suggested it before18:05
wikingyes18:05
wikingi've already given u an implementation yesterday18:05
sonne|workblackburn1: I am talking about the 'new' sgvector plan I have today18:05
wikingahhaha18:05
blackburn1ahh18:05
wiking:>18:06
wikingevolving day-by-day18:06
sonne|workblackburn1: lets have a SGVector and SGIVector18:06
blackburn1lol18:06
sonne|work(one for interface and one w/o)18:06
sonne|workblackburn1: like you suggested18:06
blackburn1sonne|work: we can use it by same interface I think18:06
sonne|workso we even remove the do_free flag from SGVector18:06
blackburn1or do you prefer explicit?18:06
sonne|workand we only use this to exchange vectors/matrices that *never* have to be freed18:07
sonne|workotherwise we use the SGIStuff18:07
blackburn1yes it is the idea I came with *proudly*18:07
blackburn1:D18:07
sonne|workzero overhead18:07
sonne|workblackburn1: this is not put in stone yet18:08
blackburn1sonne|work: yes and my suggestion is to mimic or use valarray18:08
-!- emrecelikten [~Anubis@92.44.165.109] has joined #shogun18:08
sonne|workso migration plan could be to rename all SGVector -> SGIVector18:08
sonne|workthen introduce SGVector with minimal functionality18:08
blackburn1afaik it avoids memory alignment and SSE/SSE2/someothershit optimized18:08
sonne|workand derive SGIVector from that18:08
sonne|workthen add the SGObject stuff we had before18:09
sonne|workrefcounting18:09
blackburn1works for me18:09
sonne|workand a I_don't_use_refcounting flag18:09
wikingand the day has come to use crossval of shogun ;)18:10
sonne|workthen we gradually have to check each and every occurrence of SGIVector and use SGVector if it does just sth like compute matrix18:10
sonne|workkernel matrix18:10
sonne|workand getting elements to vector18:10
blackburn1wiking: lasst xmas I used to xval18:10
wikinghow did it feel? :)18:10
blackburn1but the very next day it gave segfault me away18:10
blackburn1(I am singing)18:10
wikinghah :)18:10
blackburn1this sounds like a gay song though18:11
blackburn1I should double-check if it is allright18:11
wikingi have a new data set and no train/test split so gotta try18:11
sonne|workif we recognize there is no single use for SGVector any longer then we rename SGIVector back and end of discussion18:11
blackburn1wiking: yes and teach me how to use it18:11
wikingheheheh :)18:11
blackburn1sonne|work: sounds like a good plan actually18:11
sonne|workblackburn1: OK?18:11
blackburn1sonne|work: painful but makes sense probably18:12
blackburn1however why not to hide realization stuff?18:12
blackburn1SGVector can be an interface18:12
sonne|workand?18:13
blackburn1sonne|work: do we really need to know whether it is do_free vector or not?18:13
blackburn1in code18:14
sonne|workno18:14
sonne|workbut I am not sure what you want to tell me18:14
blackburn1sonne|work: http://yuml.me/diagram/scruffy;/class///%20Cool%20Class%20Diagram,%20%5BSGVector%5D-%5BSGInternalVector%5D,%20%5BSGVector%5D-%5BSGInterfaceVector%5D.png18:15
blackburn1but rather not http://yuml.me/diagram/scruffy;/class///%20Cool%20Class%20Diagram,%20%5BSGVector%5D-%5BSGInterfaceVector%5D.png18:16
sonne|workblackburn1: why that?18:17
sonne|workall the functions like add / subset etc are in SGVector18:17
sonne|workso Internal would be the same as SGVector18:18
blackburn1sonne|work: hmm I think it would be more extensible18:19
blackburn1who knows may be we will come with LazyVector :D18:19
sonne|workblackburn1: recall that we cannot have a single virtual function18:20
sonne|work(in SGVector)18:20
blackburn1why?18:20
sonne|workhuge overhead18:20
sonne|workvtable needs to be stored...18:20
blackburn1sonne|work: but how would you like to handle interface vector?18:21
sonne|workIVec will have virtual functions and overhead crap18:21
blackburn1sonne|work: hmm but will be independent from vec?18:21
sonne|workno18:23
blackburn1sonne|work: what will they share then?18:23
sonne|workall functions18:23
blackburn1okay got you18:24
blackburn1sonne|work: I know you do some java - you should understand why I want some inheritance there ;)18:24
blackburn1recall how rich Collection is18:25
blackburn1IndirectList is a great example how good is to have general interface18:25
sonne|workblackburn1: I know18:26
sonne|workbut this is all about saving every single bit possible :)18:26
blackburn1yeah18:26
sonne|workotherwise we could just use java18:26
sonne|workand no problems like we have now but only with GC :D18:26
blackburn1sonne|work: yeah it would be much easier to handle design and GC in java18:28
* sonne|work is happy to not have to use java at times18:29
sonne|workI am fighting with overoverover head here18:30
blackburn1sonne|work: really? I used to work on soft that would be infeasible in C++ so I don't really know the overhead18:30
sonne|worka double[]  is huge compared to what we have with sgvector18:31
blackburn1it should be not much slower18:32
sonne|workobjects have a huge overhead18:32
sonne|work*HUGE*18:32
blackburn11.5x?18:32
sonne|worknot speed18:32
blackburn1overhead in memory means?18:32
sonne|workyes18:32
sonne|workit is all only about that18:32
sonne|workspeed wise java is mostly ok18:32
sonne|work(nowadays)18:32
sonne|workand if you don't have time critical stuff18:32
sonne|work(GC ...)18:32
blackburn1sonne|work: I have no idea, how big is double[] for N elements?18:33
sonne|workfor N=1 is what is relevant here18:33
sonne|workit has some name, refernce count, whatnot18:34
sonne|workthere is no sizeof() in java for a reason :P18:34
blackburn1sonne|work: there is a way still - I thought you measured ;)18:34
blackburn18 bytes for the type pointer.4 bytes for the array length.8 bytes for each element in the array (these are pointers to the actual objects).18:35
blackburn1via stackoverflow18:35
blackburn1sonne|work: not that bad18:35
sonne|workno way18:35
sonne|workthere is more18:36
sonne|workotherwise GC could not work18:36
sonne|workgtg18:36
sonne|workcu18:36
-!- sonne|work [~sonnenbu@194.78.35.195] has left #shogun []18:36
blackburn1oh the thing I forgot is to eat something18:39
blackburn1that shouldn't happen18:39
blackburn1wiking: I didn't know you are married18:39
blackburn1;)18:39
wikingok19:03
wikingi've ran a benchmarking between19:03
wikingvalarray and float vector dot product19:03
wikingok it's biased i'm redoing it19:04
-!- PhilTillet [~Philippe@2001:660:3203:402:9c89:60e6:842d:5044] has joined #shogun19:08
blackburn1wiking: would be interesting to see results19:37
blackburn1sonney2k: we discussed a little19:45
blackburn1sonney2k: I think we need leading dimension stuff like in blas19:47
blackburn1with it we can support to slice rows and cols of matrices19:49
blackburn1i.e. vector containing elements one by one19:49
blackburn1and with skip of 1019:49
-!- karlnapf [~heiko@host86-182-161-144.range86-182.btcentralplus.com] has joined #shogun19:56
wikinglol20:02
wikingi've been invited for being a chair of a session on a conference20:02
blackburn1wiking: congrats ;)20:02
emreceliktenLike this? http://26.media.tumblr.com/tumblr_ls3g7yjipZ1qegu8do1_500.png :P20:05
wikingnow let's get back to that benchmarking... i was cooking till now20:05
wikingblackburn1: hehehe and yes i'm married... i'm old man :P20:05
wikingemrecelikten: :DDD20:05
wikingexactly20:05
emrecelikten:D20:06
wikingis there an easy way to flush cache? :)20:06
PhilTilletwiking, god.my_computer.cache.flush() ; :p20:08
wiking:))20:08
PhilTillet#include <god>20:08
PhilTilletXD20:08
PhilTilletthat would be so convenient20:08
wikingcacheflush20:08
wikingon linux apparently20:08
blackburn1sonney2k: we stuckeed20:11
blackburn1why do we need SGIVector?20:11
blackburn1it always has been copied and there is no need to handle references20:11
-!- gsomix [~gsomix@188.168.5.102] has quit [Ping timeout: 252 seconds]21:30
-!- PhilTillet [~Philippe@2001:660:3203:402:9c89:60e6:842d:5044] has quit [Ping timeout: 245 seconds]21:35
-!- PhilTillet [~Philippe@157.159.47.30] has joined #shogun21:51
-!- gsomix [~gsomix@83.234.54.138] has joined #shogun22:38
wikingmmm right where was is? :)22:51
wiking*i22:51
wiking:>22:51
-!- karlnapf [~heiko@host86-182-161-144.range86-182.btcentralplus.com] has quit [Ping timeout: 276 seconds]22:57
-!- gsomix [~gsomix@83.234.54.138] has quit [Ping timeout: 265 seconds]23:00
@sonney2kblackburn1, to be able to share vectors / matrices23:03
blackburn1sonney2k: hmmm what is the case?23:03
@sonney2kblackburn1, e.g. returning the label vector multiple times23:03
blackburn1oh23:04
@sonney2kor the SGMatrix of simplefeatures23:04
blackburn1finally I understand23:04
@sonney2ketc23:04
blackburn1sonney2k: but what to do in case like23:04
blackburn1kernel method23:04
blackburn1where we get vector23:04
@sonney2kcurrently we can return it once but have to assume that the object still exists when using it23:05
@sonney2kblackburn1, in kernel method we usually get a column vector from some sgmatrix23:05
@sonney2kso it is just a ptr into sgmatrix and no interface23:05
@sonney2keverything stored as member variable should probably be sgivector23:06
blackburn1sonney2k: btw do you like idea of that loop increment?23:06
@sonney2ksuch that it can be returned23:06
@sonney2kloop increment?23:06
blackburn1sonney2k: recall blas ddot23:06
@sonney2kyou mean slicing?23:07
blackburn1ddot(N,x,1,y,1) typically right?23:07
blackburn1but if we need column row23:07
blackburn1we currently copy23:07
@sonney2kso slicing23:07
blackburn1when we can use these 'skips'23:07
blackburn1slicing sounds inconvenient but you get me23:07
blackburn1it costs one more integer though23:08
@sonney2kthe a[:,i] thing in python numpy23:08
@sonney2knot good for core sgvector then23:08
blackburn1if memory is like 1 2 3 4 5 6 7 8 923:08
blackburn1column is array+i23:09
blackburn1and row is array+3i23:09
@sonney2kI got you but still23:09
@sonney2kno overhead in sgvector23:09
@sonney2kI am certainly ok having some fat sgivector supporting that23:09
@sonney2kbtw, there are still some problems23:09
blackburn1sonney2k: that would enable us to use column rows efficiently23:10
@sonney2ksimplefeatures don't have to be in memory23:10
@sonney2kand they have preprocessors23:10
@sonney2kso get_feature_vector()23:11
@sonney2kcould potentially return a processed copy that needs to be freed23:11
blackburn1yes who handles it?23:11
@sonney2kwe used to have a free_feature_vector() function23:11
@sonney2kthat is to be called23:11
blackburn1yes23:12
blackburn1what is our plan then?23:12
@sonney2kit would have been better to have that done via some SG_UNREF(vector)23:12
@sonney2kbut then we would need SGIVector here23:12
@sonney2kso we would have quite a bit of overhead if the matrix is in memory and there are no preprocessors23:13
blackburn1sonney2k: we would need SGIVector everywhere23:13
@sonney2kwhy that?23:13
blackburn1what is the case w/o SGI?23:13
@sonney2kI can imagine that we have two functions returning once the sgvector23:13
@sonney2kand once sgivector23:13
@sonney2kdepending on what you intend to do23:13
* sonney2k runs out of battery23:13
blackburn1sonney2k: okay lets continue tomorrow or so23:14
@sonney2k3 minutes left23:14
blackburn1do you have some indication of that?23:14
@sonney2kblackburn1, you mean we don't need some efficient sgvector?23:14
@sonney2konly sgivector stuff?23:15
-!- gsomix [~gsomix@188.168.13.227] has joined #shogun23:15
blackburn1sonney2k: I don't know the case we can use it23:15
blackburn1at some point we need some interfacing and badaboom23:15
@sonney2kblackburn1, well for example one could use it for simple features23:15
@sonney2kand jsut to access labels23:15
@sonney2ketc23:15
blackburn1yes but only a few cases and it looks like we can have overhead there23:16
blackburn1I am only afraid of get_feature_vector and similar methods overhead23:16
@sonney2kI don't know - if you want to compute kernel matrix it can become costly23:16
@sonney2kif you just have 2d vectors23:16
@sonney2kbut have to increment refcount, and set some other variables23:17
blackburn1sonney2k: yes but how can we use it here?23:17
@sonney2kjust return SGVector23:17
blackburn1but who will handle memory things?23:17
blackburn1it looks like currently it is in a working state23:17
blackburn1free_feature_vector23:17
@sonney2kand require a call to simplefeatures->free_feature_vector23:17
@sonney2k(which is usually a nop)23:18
blackburn1hmm yes I agree here23:18
blackburn1but the separation could be hard23:18
@sonney2kof course it would be nicer if the free* stuff would be done by the vector itself23:18
blackburn1I mean we need get_feature_vector for interface23:18
@sonney2kblackburn1, well one always has to copy sgvectors in interface23:19
@sonney2ksince one cannot rely on it23:19
@sonney2khave to leave train23:19
@sonney2kcu23:19
blackburn1ah train23:19
blackburn1okay see you23:19
blackburn1I am probably heading to bed23:20
-!- PhilTillet [~Philippe@157.159.47.30] has quit [Ping timeout: 265 seconds]23:25
-!- wiking_ [~wiking@78-23-189-112.access.telenet.be] has joined #shogun23:28
-!- wiking_ [~wiking@78-23-189-112.access.telenet.be] has quit [Changing host]23:28
-!- wiking_ [~wiking@huwico/staff/wiking] has joined #shogun23:28
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 260 seconds]23:32
-!- wiking_ is now known as wiking23:32
-!- blackburn1 [~qdrgsm@188.168.13.62] has quit [Quit: Leaving.]23:33
@sonney2kRe23:37
@sonney2khmmhh23:38
gsomixgood night guys23:50
-!- PhilTillet [~Philippe@npasserelle10.minet.net] has joined #shogun23:53
--- Log closed Sat Apr 28 00:00:17 2012