Open in new window / Try shogun cloud
--- Log opened Thu Nov 21 00:00:12 2013
--- Day changed Thu Nov 21 2013
@wikingok00:00
@wikingso nooow00:00
@wikingbefore doing any other fucking thing in my life00:00
@wikinglet's see this swig interface w/o libshogun compilation00:00
@wiking:))))00:00
@iglesiasgthe adapters?00:01
@iglesiasgI mean, what lisitsyn  and you were talking about before00:01
@wikingnaaah00:01
@wikingit's more like00:01
@wikingcmake hacking00:01
@iglesiasgok00:01
@wikinghow to compile swig interfaces00:01
@wikingw/o compiling libshogun00:01
@iglesiasgis matlab alive btw?00:02
@wiking(using the preinstalled one)00:02
@wikingiglesiasg: dunno how to generated the mex file00:02
@wikinguntil i dont know how to do that00:02
@wikingit's not usable00:02
@iglesiasgI am afraid I will be doing lot of matlab during my phd :|00:02
@iglesiasgall right, my cmake 2.8.9 is ready now00:02
@wikingiglesiasg: y u no scikit :D00:02
@iglesiasgwiking, I think the people  where I will be are Matlab people00:03
@iglesiasgthey mentioned that in the interview00:03
@iglesiasgno idea if they will give me freedom about that though00:03
@iglesiasgbut even if they do, it is probably better that I adapt00:03
@wikingmmm00:09
@wikingmatlab man00:09
@wikingthat's like 90s00:09
@wikingand rather beginning of 90s :P00:09
@wikingbut it'snot your fault00:09
-!- travis-ci [~travis-ci@ec2-54-205-88-49.compute-1.amazonaws.com] has joined #shogun00:09
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1427882400:09
-!- travis-ci [~travis-ci@ec2-54-205-88-49.compute-1.amazonaws.com] has left #shogun []00:09
@wikingbut u should bring chaaange00:09
thoralf_Woha.  Replacing DynamicObjectArray by DynArray<void *> worked instantly.  No broken tests.00:09
@wikingbe the obama of ML00:09
@wikingthoralf_: woohooo00:09
thoralf_so maybe replacing void by StructuredLabels, but should work fine.00:10
@wikingi mean seriously00:10
thoralf_Okay, will continue tomorrow.00:10
@wikingwe could start using std::list00:10
thoralf_:)00:10
@wikingit's not more bloated than this00:10
@wiking:P00:10
@iglesiasgwtf my Shogun doesn't compile now00:11
@iglesiasgProtobufFile.h00:11
@wikinghahahah00:11
@wikingok00:11
@wikingdo u really use protobuf?00:11
@wikingor just happen to have the library around?00:11
@iglesiasgthoralf_, I add the doc fix and will change the set_label maybe tom00:11
@wikingif not using it00:11
@wikingapt-get remove the libprotobuf-dev00:12
@wikingor whatever it is called00:12
@iglesiasgit might be here because of GraphLab00:12
@wikingor comment out the protobuf detection in CMakeLists.txt00:12
@wikingiglesiasg: comment then00:12
@iglesiasgand I shouldn't remove it then until my thesis is graded  :D00:12
thoralf_iglesiasg: Don't do anything except the comment :P00:12
@iglesiasgcould screw up new experiments if I need00:13
* thoralf_ just refactors StructuredLabels00:13
@iglesiasgthoralf_, you are the man :)00:13
@wikingiglesiasg: comment out line 741-748 in CMakeLists.txt00:13
@iglesiasgI just wrote added to pre-allocate to "number of labels to pre-allocate"00:13
@iglesiasgwiking, it is ok. I will try tomorrow again in the office computer00:14
@iglesiasgthis one takes too much time to compile :(00:14
@wikingiglesiasg: okok i'm just saying00:14
@wikingit's not a surprise it doesn't work00:14
@wikingwe fucking broke develop atm00:14
@wiking;P00:14
@wikingbut people can have fun with 3.000:14
@iglesiasgI should go back to arch again soon00:14
@wikingyeah arch is cool00:15
@iglesiasgI have the feeling everything is much much slower with ubuntu00:15
@wikingi[m using arch on my raspi00:15
@iglesiasgI don't know if it even makes sense00:15
@iglesiasgoh nice00:15
@iglesiasgI have it with raspbian00:15
@wikingaaah outdated shiatz00:15
@iglesiasgdoing ros stuff with it00:15
@iglesiasgand I already tried once to get ros in arch00:15
@wikingarch has much more updated packages00:15
@iglesiasgand was not fun00:15
@wikinghahehhehe00:15
@wikingwell tit for tat00:16
@iglesiasghehe yeah00:16
@wikingok00:16
@wikingfuckit00:16
@wikingi'm gonna masacre now the cmakefile00:16
@wikingagaaain00:16
shogun-notifier-shogun: Fernando Iglesias :develop * 9079856 / src/shogun/labels/StructuredLabels.h: https://github.com/shogun-toolbox/shogun/commit/90798560260a15596670f69529397fd1f564e85d00:16
shogun-notifier-shogun: Minor doc improvement in StructuredLabels constructor00:16
shogun-notifier-shogun: Fernando Iglesias :develop * 46e1bd5 / src/shogun/labels/StructuredLabels.h: https://github.com/shogun-toolbox/shogun/commit/46e1bd566a72d909d03722278de1a303e1e2cfb200:16
shogun-notifier-shogun: Merge pull request #1761 from iglesias/develop00:16
shogun-notifier-shogun:00:16
shogun-notifier-shogun: Minor doc improvement in StructuredLabels constructor00:16
@iglesiasgthat was just a doc change so it should not kill anything :D00:17
@iglesiasgall right guys00:18
@iglesiasggood night00:18
@iglesiasgit was a pleasure00:18
-!- iglesiasg [~iglesiasg@s83-179-44-135.cust.tele2.se] has quit [Quit: Leaving]00:18
shogun-buildbot_build #2036 of deb3 - modular_interfaces is complete: Failure [failed test ruby modular]  Build details are at http://buildbot.shogun-toolbox.org/builders/deb3%20-%20modular_interfaces/builds/2036  blamelist: Soeren Sonnenburg <sonne@debian.org>00:19
shogun-buildbot_build #1961 of bsd1 - libshogun is complete: Failure [failed compile test]  Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/1961  blamelist: Soeren Sonnenburg <sonne@debian.org>, Fernando Iglesias <fernando.iglesiasg@gmail.com>00:30
thoralf_src/shogun/base/class_list.cpp:673:125: error: cannot convert 'shogun::CFactorGraphObservation*' to 'shogun::CSGObject*' in return00:43
thoralf_Grr.00:43
thoralf_Dependencies on CSGObject are hidden in several places. ;)00:44
thoralf_wiking: Any idea how to get StructuredData subclasses out of class_list?00:46
thoralf_Houston, we've lost Serialization on StructuredData.00:50
thoralf_total heap usage: 1,000,299 allocs, 299 frees, 32,033,787 bytes allocated00:51
thoralf_32 Bytes/Float.00:51
thoralf_wiking: Editing class_list fixed it... is there a blacklist I could extend?00:53
-!- thoralf_ [~thoralf@91-66-33-4-dynip.superkabel.de] has quit [Quit: Konversation terminated!]00:53
@wikingyes00:54
@wikingor i think so00:54
@wikingsrc/shogun/base/class_list.cpp.py00:55
@wikingcheck this00:55
-!- hushell [~hushell@c-50-188-141-210.hsd1.or.comcast.net] has quit [Ping timeout: 272 seconds]01:12
-!- hushell [~hushell@c-50-188-141-210.hsd1.or.comcast.net] has joined #shogun01:15
shogun-buildbot_build #2037 of deb3 - modular_interfaces is complete: Success [build successful]  Build details are at http://buildbot.shogun-toolbox.org/builders/deb3%20-%20modular_interfaces/builds/203701:24
shogun-buildbot_build #123 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/123  blamelist: Soeren Sonnenburg <sonne@debian.org>, Fernando Iglesias <fernando.iglesiasg@gmail.com>01:39
shogun-buildbot_build #121 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/121  blamelist: Soeren Sonnenburg <sonne@debian.org>, Fernando Iglesias <fernando.iglesiasg@gmail.com>01:55
-!- pickle27 [~kevin@24-212-221-132.cable.teksavvy.com] has joined #shogun02:36
shogun-buildbot_build #140 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/140  blamelist: Soeren Sonnenburg <sonne@debian.org>, Fernando Iglesias <fernando.iglesiasg@gmail.com>02:38
-!- sonne|osx [~sonne@f053043202.adsl.alicedsl.de] has quit [Ping timeout: 248 seconds]03:14
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]03:16
shogun-buildbot_build #624 of nightly_default is complete: Failure [failed notebooks]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/62404:59
-!- pickle27 [~kevin@24-212-221-132.cable.teksavvy.com] has quit [Quit: Leaving.]05:04
-!- zxtx [~zv@129-79-241-148.dhcp-bl.indiana.edu] has quit [Ping timeout: 245 seconds]06:16
-!- zxtx [~zv@c-98-193-83-24.hsd1.il.comcast.net] has joined #shogun06:56
-!- Saurabh7 [~Saurabh7@115.248.130.148] has joined #shogun06:59
-!- Jagged [~NULL@vtluug/member/jagged] has quit [Read error: Operation timed out]07:38
-!- Jagged [~NULL@vtluug/member/jagged] has joined #shogun07:38
-!- Saurabh7 [~Saurabh7@115.248.130.148] has left #shogun ["Leaving"]07:45
-!- lisitsyn1 [~lisitsin@mxs.kg.ru] has joined #shogun08:53
lisitsyn1ah again08:53
lisitsyn1sorry guys for gunfire haha08:53
-!- lisitsyn [~lisitsyn@80.252.20.67] has quit [Disconnected by services]08:53
-!- lisitsyn1 is now known as lisitsyn08:54
-!- mode/#shogun [+o lisitsyn] by ChanServ08:54
-!- lisitsyn1 [~lisitsyn@80.252.20.67] has joined #shogun08:54
-!- lisitsyn1 was kicked from #shogun by lisitsyn [lisitsyn1]09:07
-!- iglesiasg [~iglesiasg@s83-179-44-135.cust.tele2.se] has joined #shogun09:21
-!- mode/#shogun [+o iglesiasg] by ChanServ09:21
@wikinglisitsyn: btw about the droping stuff: it's in a way already part of shogun.. check the computational part. there's a Run class where u could plugin basically anything u want09:23
@wikingbut what really concerns me atm is this RealNumber bloated shit09:24
@wikingthat we need 127x more space for storing a stupid double :P09:24
@lisitsynhaha09:26
@iglesiasgc'mon RealNumber is not be used for real09:28
@wikingiglesiasg: but essentially yes09:28
@iglesiasgwhat do you mean?09:29
@wikingwell what do u store in RealNumber09:29
@wikinga double no?09:29
@wikinganything else?09:29
@iglesiasgyep, just a double09:29
@iglesiasgbut RealNumber is just used in SO09:30
@wikingyeah i know09:30
@wikingbut this is outrages09:30
@wikingoutragous09:30
@iglesiasghaha ok09:30
@wikingi mean this is far from being optimal09:30
@iglesiasgbut it is not used!09:30
@iglesiasglet's care about real things09:30
@wikingiglesiasg: thoralf is using it no? :)09:30
@iglesiasgwiking, no no09:30
@iglesiasgthoralf had the same problem but using other kind of labels09:31
@iglesiasgsince the overhead is the same09:31
@iglesiasgor at least I understood that09:31
@wikingiglesiasg: anyhow i think we should do something with this huge overhead somehow09:31
@wikingif possible at all09:31
@iglesiasgit should be less now after the shrinking of the DynamicObjectArray09:32
@iglesiasgalthough I am not sure if just setting size 4 for all Parameter and ParameterMap is the best way to do this09:32
@wikingbesser82: do u really dislike the idea of vpn between the shoung-ca and shogun's fatbot?09:32
thoralfiglesiasg, wiking: Heard my name? ;)10:25
* thoralf could change StructuredData to inherit from RefCount, nothing else.10:30
thoralfAny objections?10:30
thoralfSeems to work.10:30
thoralfNo more Parameter(Map) objects for each Label.10:31
thoralfOnly 28 bytes overhead/label10:31
besser82wiking: not really, if vpn is the way: just do it ;)10:40
besser82wiking: I just prefer ssh, because it's less cpu-comsuming10:41
besser82wiking: There is no other reason, against vpn.  Just take the easier and working approach  :D10:42
besser82sonney2k_: I'm heading down to get the cmake stuff running with the protobuf-thing and the use-procompiled-shogun-lib10:43
@wikingbesser82: i was looking into this and i've seen that we should create ShogunConfig.cmake and ShogunConfigVersion.cake10:43
@wiking*cmake10:44
besser82sonney2k_: It simply took longer, because I got some unforeseen job to accomplish and stuff10:44
besser82wiking: I know my approach uses some ShogunConfig.cmake, introduces shogun.pc and shogun_config.h10:44
@wikingbesser82: we already have config.h10:45
@wikingmaybe that needs fixing10:45
besser82wiking: as far as I've seen, yes10:45
besser82wiking: because there are lot of definitions passed by cmake to the compiler, which should be in config.h and used from there...#10:46
@wikingbesser82: go ahead with fixess :P10:47
besser82wiking: and that config.h which is actually generated, gets generated from some external script, like version.h10:47
besser82wiking: which both should be generated when running cmake-configure-stage10:47
besser82wiking: instead of `make all`10:48
@wikingnot the version.h10:48
besser82wiking: ???10:48
besser82wiking: what#s the problem with that?10:48
besser82wiking: everything the autogen-script does can be done from cmake as well10:49
@wikingwell version.h should be generated compile time10:49
@wikingand not configure time10:49
besser82wiking: any special reason for that?10:49
@wikingwell becasue the git rev version number can change10:50
@wikingafter configure step10:50
@wikingobviously10:50
@wikingbesser82: but this is already working like this in shogun10:50
@wikingso this is ok10:50
@wikingi mean version.h10:50
besser82wiking: allright.10:51
besser82wiking: but how the git-sha can change after configuring??? *confused*10:51
@wikingbesser82: check this10:51
@wikingcmake/version.cmake10:51
besser82wiking: will dive into...10:52
-!- iglesiasg [~iglesiasg@s83-179-44-135.cust.tele2.se] has quit [Quit: Leaving]10:57
-!- iglesiasg [~iglesias@2001:6b0:1:1da0:d5d3:7e38:9901:108c] has joined #shogun12:52
-!- mode/#shogun [+o iglesiasg] by ChanServ12:52
@iglesiasghi thoralf12:52
@iglesiasgI had to run before so I didn't answer, sorry about that12:52
@iglesiasgthoralf,  so at first sight what I see is that we would have to change the CDynamicObjectArray to a DynArray or so12:55
@iglesiasgbut that should be fine12:55
-!- sonne|osx [~sonne@f053047007.adsl.alicedsl.de] has joined #shogun13:17
@wikingsonne|osx: bazinga13:18
sonne|osxwiking: good news everyone13:19
sonne|osxwhats up?13:19
@wikingnotmuhc13:21
@wikingw8ing for besser8213:21
thoralfiglesiasg: I already replaced it by DynArray, but it's not that trivial because (1.) CDynamicObjectArray took care about SG_REF/UNREF13:26
@iglesiasgthoralf, does RefCount clean memory automatically?13:27
thoralfiglesiasg: and (2.) it seems that it's again consuming 5k/instance, but I need to check that.13:27
@iglesiasgthoralf, otherwise, what about using SGReferencedData instead13:27
thoralfiglesiasg: It does not.  It's just a thread safe counter class.13:27
thoralfsonne|osx: Did you revert your changes?  DynArray again allocates 5*1024 bytes for each label...13:37
sonne|osxthoralf: I understand why but now how to fix it - did it compile for you last time (buildbot was unhappy so I tried to do it differently but failed obviously)13:38
thoralfsonne|osx: you did something like "DynArray<xxx> foo(4);" -- my compiler complained, so I changed it locally to something like "DynArray<xxx> foo = DynArray<xxx>(4);", which worked well.13:41
thoralfI did this while watching a movie, so I didn't check in depth.13:42
sonne|osxthoralf: ahh ok then. maybe I should just do it this way then - or you do it if you have time now (search for set_granularity in the base/Param*.cpp)13:42
thoralfI'll try.13:46
thoralfset_granularity(int32_t g)13:46
thoralfg=CMath::max(g,128);13:46
thoralfLOL.13:46
sonne|osxhuh? why that stupid limit?13:49
sonne|osxjust drop it13:49
sonne|osxI mean 113:50
sonne|osxshould be the limit13:50
thoralfOkay.13:51
@iglesiasgI think that a method to shrink the space is better than setting the granularity in Parameter13:53
@iglesiasgif for some reason in a Parameter object lot of elements are added to the list13:54
@iglesiasgit would be very slow if the granularity is that small13:54
@wikingheheh lol13:56
@wikingiglesiasg: have u read the mailinglist?13:56
@iglesiasgPrimalMosek?13:56
@wikingyeps13:56
@wikingi wonder wtf is happening there13:57
@iglesiasgnot yet13:57
@wikingbtw13:57
@wikingdo we support svmlight input format -> SOSVM multiclass?13:57
@iglesiasgI haven't used that so AFAIK no13:57
@iglesiasgbut it might be13:57
thoralfiglesiasg: I think we should talk about "preallocated size on construction" instead of granularity...14:03
thoralfiglesiasg: In most cases we either (a) need only small arrays or (b) we know in advance that it's going to be BIIIG.14:03
@iglesiasgthoralf, well not really14:04
@iglesiasgthoralf, that is the whole concept of dynamic arrays14:04
@iglesiasgthoralf, like dynamic resizing arrays. You can get amortized linear time insertion without even giving a preallocated size14:05
thoralfCan you achieve linear size by only resizing a fixed number of elements?14:06
@iglesiasgwhat I mean with this is that IMHO is OK to talk about granularity here14:06
thoralfShouldn't something exponential happen wrt. the size happen?14:06
@iglesiasgthoralf, what? linear time you mean?14:06
thoralfiglesiasg: For having linear time, you need to multiply the array size.14:07
@iglesiasgyes14:07
thoralfBut you're only adding K elements, for fixed K.14:07
thoralfSo no linear time anyway.14:07
@iglesiasgSo if this granularity is not set, it is just set to a constant value?14:08
thoralf128 per default14:08
thoralfBut this is not the point.14:08
@iglesiasgok14:08
@iglesiasgwhat is it then? :)14:09
thoralfThe point is: why something so expensive on every SGObject creation.14:09
thoralfThe rest is only micro-optimization.14:09
@iglesiasgyes, that is true14:09
@iglesiasgprobably Heiko has a good reason related to CV and model selection to use this Parameter attributes in SGObject14:09
thoralfOr the point might be: Why does StructuredData extend SGObject14:09
@iglesiasgI have nothing against making StructuredData not extend SGObject14:10
thoralfI tried it, but since serialization is encoded in the class name (CStructuredData), it makes a lot of pain to change this.14:11
thoralfAnd we're losing SG_REF/UNREF.14:12
@iglesiasgthoralf, I don't see the serialization point, let me check the code14:14
thoralfclass_list.cpp.py14:15
thoralfAutomagic stuff.14:15
thoralfclass_list.cpp breaks as soon as you remove SGObject from StructData.14:15
@iglesiasgok so pretty much the thing is that we cannot serialize CStructuredData if it is not SGObject14:15
@iglesiasgthoralf, so there are a couple of reasong to keep it as SGObject14:17
@iglesiasgthoralf, but the problem is that it takes too much memory14:17
@iglesiasgthoralf, what is the inconvenient with shrinking the memory used by the Parameter attributes in SGObject14:17
@iglesiasgI don't mean it to do *everywhere*14:17
@iglesiasgfor instance, right now to me it seems reasonable to do it in the CStructuredData constructor14:18
thoralfThe memory will be allocated by the default constructor of DynArray.14:19
thoralfSo shrinking needs to realloc and eventually copy stuff around.14:19
thoralfI will commit something that initializes Parameter/ParameterMap with smaller sizes, as a hot-fix.14:20
thoralf160 Bytes instead of 5KiB14:20
@iglesiasgok14:20
@iglesiasgthoralf, out of curiosity, when "shrinking needs to realloc and eventually copy stuff around", is that overhead relevant?14:21
thoralfI think that (re)alloc is expensive and yields to memory fragmentation.  But it's shouldn't be relevant.14:23
thoralfiglesiasg: Have a look: https://github.com/shogun-toolbox/shogun/pull/176214:23
thoralfsonne|osx as well14:23
@iglesiasgthoralf, yep, I opened it already. Looks good to me, let's see how does it looks for travis14:25
thoralfNote that I didn't change the default size.14:25
thoralfOnly for usages within Parameter(Map).14:25
-!- zxtx [~zv@c-98-193-83-24.hsd1.il.comcast.net] has quit [Ping timeout: 272 seconds]14:26
thoralfOnly 508 bytes per float now. :D14:26
thoralftotal heap usage: 11,000,328 allocs, 11,000,328 frees, 508,034,974 bytes allocated14:27
thoralf1M entries.14:27
thoralfSo each label needs 11 (!) mallocs and uses 508 bytes.14:27
@iglesiasgthoralf, what labels are you exactly using btw?14:28
thoralfMultilabels14:28
thoralfNot yet committed.14:28
@iglesiasgwhat are they exactly?14:28
@iglesiasga set of ints?14:28
thoralfA vector of {0,1}^K14:28
thoralfFormally.14:28
@iglesiasgall right14:28
thoralfImplemented as a set of ints.14:28
thoralfTo make it sparse. :)14:28
thoralfto exploit sparsity14:29
@iglesiasgis your K large?14:29
thoralf700014:29
@iglesiasgthen it is good to make it sparse if labels are going to have to 1 just a few entries :)14:30
@iglesiasggood good14:30
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun15:04
shogun-notifier-shogun: Thoralf Klein :develop * 72f0bb1 / src/shogun/base/ (3 files): https://github.com/shogun-toolbox/shogun/commit/72f0bb15699f5b5abb90e49b045b97580ce803a915:04
shogun-notifier-shogun: Fixed initialization of DynArray in Parameter/ParameterMap to get memory footprint small:15:04
shogun-notifier-shogun: DynArray will be instanciated 5 times for each SGObject (via Parameter/ParameterMap)15:04
shogun-notifier-shogun: which introduced a memory overhead of 5kb per instance.  With this commit, the overhead15:04
shogun-notifier-shogun: should be reduced to 160 bytes.15:04
shogun-notifier-shogun: Fernando Iglesias :develop * fca253d / src/shogun/base/ (3 files): https://github.com/shogun-toolbox/shogun/commit/fca253db53149817a2fcdf5ba584d92366de11dd15:04
shogun-notifier-shogun: Merge pull request #1762 from tklein23/develop15:04
shogun-notifier-shogun:15:04
shogun-notifier-shogun: Fixed initialization of DynArray in Parameter/ParameterMap to reduce memory footprint15:04
shogun-notifier-shogun: Soeren Sonnenburg :develop * 22ba728 / src/shogun/base/DynArray.h: https://github.com/shogun-toolbox/shogun/commit/22ba728482a7adc4cef586e1b0cf84348a7ac90015:05
shogun-notifier-shogun: 1 should be the lower bound15:05
sonne|osxthoralf: you could even use one - would make sense IMHO15:05
sonne|osxthoralf: I mean this is just done once for <10 parameters right so no reason to try to be fast15:06
shogun-notifier-shogun: Fernando Iglesias :develop * ba1a837 / src/shogun/base/DynArray.h: https://github.com/shogun-toolbox/shogun/commit/ba1a8375d55524f20d353c1ca333931aa38d083315:07
shogun-notifier-shogun: Set to 1 granulatity minimum threshold in DynArray::set_granularity15:07
shogun-notifier-shogun: Fernando Iglesias :develop * d8aab2f / : https://github.com/shogun-toolbox/shogun/commit/d8aab2fc464f0fc018aac9761d76404d1372155515:07
shogun-notifier-shogun: Merge pull request #1763 from iglesias/develop15:07
shogun-notifier-shogun:15:07
shogun-notifier-shogun: Set to 1 granulatity minimum threshold in DynArray::set_granularity15:07
thoralfsonne|osx: Yeah, but the memory overhead for small memory chunks is quite big, so it usually makes no difference if you allocate 1 or 20 bytes.15:08
sonne|osxthoralf: it is still 3*8 bytes wasted * 415:10
thoralfOkay, let me check how many elements the array holds in average. ;)15:10
thoralfMicro-optimization done right[tm] ;)15:11
sonne|osxthoralf: why even try? I mean this is like 10 parameters / class max15:11
sonne|osxso I guess we usually have 0-2 parameters15:11
sonne|osxthoralf: but you know that the array is used here for convenience not optimization right?15:12
sonne|osxthoralf: so that one can do SG_ADD( some parameter)15:12
sonne|osxthoralf: anyway SGObjects are not small so one should not use them if you intend to have  very little payload15:14
sonne|osxthey are just not meant for that15:14
sonne|osxone should rather use aggregated data types15:14
sonne|osxI mean storing a real number as some CRealNumber is waste of resources15:15
sonne|osxCRealVector or CRealMatrix or whatever is what one should use15:15
thoralfsonne|osx: I'm not storing floats in StructuredData.  It's complex labels, but even in my case the overhead exceeded the payload by many orders.15:16
thoralfcomplex as-in not-trivial15:16
sonne|osxthen CSGObjects are not waht you want15:17
thoralfBtw., the average size of the DynArrays is 0 for StructuredData ;)15:17
sonne|osxthey have 8 pointers several bools15:18
sonne|osxno size optimization at all15:18
thoralfsonne|osx: "then CSGObjects are not waht you want" <-- That's right, but as I was using the SO framework of iglesias, I didn't have a choice.15:18
sonne|osxso the right fix would be that StructuredData was not derived from SGObject15:19
thoralfYes.15:19
thoralfI tried this, but it's not trivial.15:19
sonne|osxquestion is what of SGObject does it need15:19
sonne|osxiglesiasg: ?15:19
thoralfFirst, compiling class_list.cpp failed, but I blacklisted this.15:19
@iglesiasgI think only REF and UNREF is actually used15:19
thoralfSecond, You have to care about REF/UNREF15:20
@iglesiasgand serialization too according to thoralf15:20
thoralfIt helps te rename the classes and remove the "C".15:20
thoralfBut still: Memory will not be freed automatically.15:21
sonne|osxiglesiasg: well then use delete instead of unref and good? or is this an object shared among variius places?15:21
sonne|osxthoralf: yes rule is - classes prefixed with C - are CSGObjects and will be serialized15:22
sonne|osxetc15:22
@iglesiasgsonne|osx, where I have used it it was not shared15:22
sonne|osxiglesiasg: so then easy fix right?15:22
@iglesiasgyes, I think so15:22
shogun-buildbot_build #1962 of bsd1 - libshogun is complete: Failure [failed compile test]  Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/1962  blamelist: Thoralf Klein <thoralf.klein@zib.de>15:22
thoralfiglesiasg: We have to take care a bit, since StructeredData is passed around in SO framework (argmax, etc.)15:24
thoralfBut it should help to take care and run valgrind a few times.15:24
sonne|osxthoralf: I guess igleasias knows where it is initally set - in this object it has to be destroyed15:25
@iglesiasgit would basically be in the destruction of the labels15:26
thoralfYes, I think so.15:26
sonne|osxiglesiasg, thoralf and when you set the object you have to delete the old one (make sure that the you do an init with NULL ...)15:29
shogun-notifier-shogun: Soeren Sonnenburg :develop * d52da29 / src/shogun/base/Parameter.cpp,src/shogun/base/ParameterMap.cpp: https://github.com/shogun-toolbox/shogun/commit/d52da29673ed829b285ebee0dc32062ce695148415:31
shogun-notifier-shogun: use 1 even15:31
thoralfDamn.  I just prepared a PR for this. ;)15:33
shogun-buildbot_build #1963 of bsd1 - libshogun is complete: Failure [failed compile test]  Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/1963  blamelist: Soeren Sonnenburg <sonne@debian.org>, Fernando Iglesias <fernando.iglesiasg@gmail.com>15:38
-!- hushell [~hushell@c-50-188-141-210.hsd1.or.comcast.net] has quit [Ping timeout: 248 seconds]15:41
@wikingis there a fast way i can do Sparsefeatures->DenseFeatureS?15:47
-!- thoralf [~thoralf@enki.zib.de] has quit [Quit: Konversation terminated!]15:50
-!- thoralf [~thoralf@enki.zib.de] has joined #shogun15:50
-!- iglesiasg [~iglesias@2001:6b0:1:1da0:d5d3:7e38:9901:108c] has quit [Quit: Ex-Chat]15:58
@wikingwe dont have a from_sparse(SGSparseMatrix) converter in SGMatrix :S16:00
-!- FSCV [~FSCV@201.161.7.110] has joined #shogun16:05
-!- lisitsyn [~lisitsin@mxs.kg.ru] has quit [Quit: Leaving.]16:09
sonne|osxwiking: do we have one in CDenseFeatures?16:48
sonne|osxwiking: actually I guess only the other way round16:49
@wikingnoup16:49
@wikingyes in indeed only one way16:49
sonne|osxwiking: well sparse -> dense is very easy :)16:49
@wikingheheh indeed16:50
@wikingnevermind16:50
@wikingi could use sparsefeatures16:50
@wikingso it was ok16:50
sonne|osxthoralf: we really should do the structureddata change16:50
thoralfsonne|osx: Of course.  Do you have an agenda for this?16:53
shogun-buildbot_build #124 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/124  blamelist: Thoralf Klein <thoralf.klein@zib.de>16:55
shogun-buildbot_build #1964 of bsd1 - libshogun is complete: Failure [failed compile test]  Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/1964  blamelist: Soeren Sonnenburg <sonne@debian.org>16:55
sonne|osxthoralf: agenda is you do it and I review the stuff :) I would prefer fixing cmake & protbuf16:56
thoralfEhrm. ;)16:57
thoralfsonne|osx: What's the exact plan for this?  Removing base class SGObject from structured stuff.  Renaming subclasses so that they don't start with C (otherwise class_list.cpp will fail).  Replacing all SG_UNREF on labels by delete.16:58
thoralfAnything else?16:59
thoralfAny expected side effects?16:59
sonne|osxthoralf: and init the data with NULL in labels and also free it before the setter is doing the set16:59
sonne|osxno side effects if this is really only *internal* data17:00
sonne|osxit cannot be exposed to e.g. python modular or so (otherwise ref/unref needs to work)17:00
shogun-buildbot_build #335 of FCRH - libshogun is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/FCRH%20-%20libshogun/builds/335  blamelist: Soeren Sonnenburg <sonne@debian.org>17:03
shogun-buildbot_build #122 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/122  blamelist: Thoralf Klein <thoralf.klein@zib.de>17:20
thoralfsonne|osx: We somehow broke serialization ^17:22
sonne|osxthoralf: you mean bug in DynArray with granularity 1?17:24
sonne|osxdon't see it yet though17:25
thoralfhttp://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/122/steps/test/logs/stdio17:25
thoralfNo, DynArray and tests are work locally.  Seems to be build-system related.17:30
sonne|osxnah these are not really tests17:36
-!- Saurabh7 [~Saurabh7@115.248.130.148] has joined #shogun17:51
shogun-buildbot_build #141 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/141  blamelist: Thoralf Klein <thoralf.klein@zib.de>18:04
@wikinglol18:06
@wikinghttp://stackoverflow.com/questions/20122267/ifinite-was-not-declared-in-this-scope18:06
@wikingstackoverflow :)18:06
@wikingafaik we should really do something about18:06
@wikingcmath and math.h18:06
@wikingand btw:18:07
@wikingMath.h:1257:11: warning: 'finite' is deprecated: first deprecated in OS X 10.9 [-Wdeprecated-declarations]18:07
@wiking                        return finite(f);18:07
@wiking                               ^18:07
@wiking/usr/include/math.h:718:12: note: 'finite' declared here18:07
@wikingextern int finite(double) __OSX_AVAILABLE_BUT_DEPRECATED(__MAC_10_0, __MAC_10_9, __IPHONE_NA, __IPHONE_NA);18:07
-!- thoralf [~thoralf@enki.zib.de] has quit [Quit: Konversation terminated!]18:13
shogun-buildbot_build #125 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/125  blamelist: Soeren Sonnenburg <sonne@debian.org>, Fernando Iglesias <fernando.iglesiasg@gmail.com>18:15
@wikingfyi we have some sorts of memory leak in ./tests/unit/unit-SGObject18:16
-!- lisitsyn1 [~lisitsyn@80.252.20.67] has joined #shogun18:21
shogun-buildbot_build #123 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/123  blamelist: Soeren Sonnenburg <sonne@debian.org>, Fernando Iglesias <fernando.iglesiasg@gmail.com>18:30
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]18:31
@wikingsonne|osx: ping.18:53
@wikingsonne|osx: is it ok if i change inline static int is_finite(double f)18:53
@wikingto static int is_finite(double f)18:53
@wikingjust as like static int is_infinity(double f);18:54
@wiking?18:54
shogun-buildbot_build #142 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/142  blamelist: Soeren Sonnenburg <sonne@debian.org>, Fernando Iglesias <fernando.iglesiasg@gmail.com>19:13
-!- FSCV [~FSCV@201.161.7.110] has quit [Quit: Leaving]19:30
@wikingbtw anybody here knows how to generate that stupid mex file for matlab static interface?19:33
shogun-buildbot_build #126 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/126  blamelist: Soeren Sonnenburg <sonne@debian.org>19:34
sonne|osxwiking: try mex19:43
sonne|osxwiking: this is_finite stuff worked already with configure no idea how to scan for it with cmake19:45
@wikingsonne|osx: is it ok if i do it like that?19:45
@wikinginline -> static?19:45
sonne|osxwiking: sure19:46
sonne|osxwiking: wow - many people with the ifinite issue :D19:46
sonne|osxand all trying to fix it?!19:46
sonne|osxoopsi19:46
sonne|osxthere is a typo19:46
sonne|osxifinite(f); -> isfinite(f)19:46
sonne|osxhmmh19:48
sonne|osxwe don't have ifinite in the code19:48
sonne|osxsomething weird is going on there...19:48
shogun-buildbot_build #124 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/124  blamelist: Soeren Sonnenburg <sonne@debian.org>19:49
sonne|osxdoesn't look like it is our fault19:51
@wikingsonne|osx: here19:52
@wikingcheck this patch19:52
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun19:53
shogun-notifier-shogun: Viktor Gal :develop * 17babf2 / / (5 files): https://github.com/shogun-toolbox/shogun/commit/17babf2deb5c022396d0503a48d7e7de93b02fc619:53
shogun-notifier-shogun: Refactor is_nan, is_infinity and is_finite in CMath19:53
shogun-notifier-shogun: Disable ProtoBuf detection19:53
@wikingsonne|osx: soon we should do a hotfix release20:07
shogun-buildbot_build #143 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/143  blamelist: Soeren Sonnenburg <sonne@debian.org>20:35
shogun-buildbot_build #1685 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at http://buildbot.shogun-toolbox.org/builders/cyg1%20-%20libshogun/builds/1685  blamelist: Viktor Gal <viktor.gal@maeth.com>20:40
shogun-buildbot_build #1965 of bsd1 - libshogun is complete: Failure [failed compile test]  Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/1965  blamelist: Viktor Gal <viktor.gal@maeth.com>20:43
@wikingdoh20:47
@wiking/usr/local/bin/ld: CMakeFiles/libshogun.dir/ui/GUIPluginEstimate.cpp.o: relocation R_X86_64_32S against `vtable for shogun::CGUIPluginEstimate' can not be used when making a shared object; recompile with -fPIC20:47
@wikingfreebsd :S20:47
@wikingfpic detection is not working20:47
-!- hushell [~hushell@c-50-188-141-210.hsd1.or.comcast.net] has joined #shogun20:52
shogun-buildbot_build #336 of FCRH - libshogun is complete: Success [build successful]  Build details are at http://buildbot.shogun-toolbox.org/builders/FCRH%20-%20libshogun/builds/33621:10
-!- zxtx [~zv@ool-2f110054.dyn.optonline.net] has joined #shogun21:26
shogun-notifier-shogun: Viktor Gal :develop * ad0f48e / CMakeLists.txt: https://github.com/shogun-toolbox/shogun/commit/ad0f48e150072e2236ab5ea0b6620014c7e33edc21:35
shogun-notifier-shogun: Include CheckCXXSourceCompiles in CMakeLists.txt21:35
shogun-buildbot_build #1966 of bsd1 - libshogun is complete: Failure [failed compile test]  Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/1966  blamelist: Viktor Gal <viktor.gal@maeth.com>22:01
shogun-buildbot_build #127 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/127  blamelist: Viktor Gal <viktor.gal@maeth.com>22:01
-!- iglesiasg [~iglesiasg@s83-179-44-135.cust.tele2.se] has joined #shogun22:07
-!- mode/#shogun [+o iglesiasg] by ChanServ22:07
shogun-buildbot_build #343 of FC19 - libshogun is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/FC19%20-%20libshogun/builds/343  blamelist: Viktor Gal <viktor.gal@maeth.com>22:09
shogun-buildbot_build #125 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/125  blamelist: Viktor Gal <viktor.gal@maeth.com>22:26
shogun-notifier-shogun: Viktor Gal :develop * 66d819f / src/shogun/structure/CCSOSVM.cpp,src/shogun/structure/CCSOSVM.h: https://github.com/shogun-toolbox/shogun/commit/66d819f1bbd3649cb2afbcdc28c0e45ba48a5aaa22:38
shogun-notifier-shogun: Small refactoring of CCSOSVM22:38
sonne|osxwiking: what is the issue with is_nan etc? it was all working OK before we switched to cmake?!22:38
sonne|osxwiking: ahh cool nice checks22:39
sonne|osxwiking: you should just merge that and then we see if the cygwin bot is happy or not22:40
@wikingwe'll see now what happens with cygwin22:41
@wikingsonne|osx: there was nothing wrong with is_nan just made it a bit more portable22:42
* wiking has a hunch that some basic mathematical function is being broken atm...22:42
@wikingthat's why ccsosvm not working anymore22:42
sonne|osxwiking: if you had tests :P22:43
@wikingsonne|osx: mosek based...so wouldnt have helped22:43
sonne|osxso useless stuff anyway22:44
@wikingwe should have unittest for sparse_dot...22:44
@wikingwtf is this that there's no gettimeofday22:56
sonne|osxwoa22:56
sonne|osxthat is the seed for the rng22:56
@wikingsonne|osx: heiko style22:58
shogun-buildbot_build #1686 of cyg1 - libshogun is complete: Success [build successful]  Build details are at http://buildbot.shogun-toolbox.org/builders/cyg1%20-%20libshogun/builds/168622:59
sonne|osxwiking: well we could fall back to time() if getttimeofday is not available22:59
sonne|osxso cyg is happy22:59
@wikingnoup22:59
@wikingNever use time() to initialize srand()22:59
@wikinggoogle for this22:59
sonne|osxwhy not23:00
sonne|osxI mean we have the pid and other stuff mixed in too23:00
@wikingread the article23:01
sonne|osxwiking: no good enough as a fall back - at least some randomness23:01
@wikingsonne|osx: /dev/urandom23:02
@wikingsonne|osx: dunno what could be on non UNIX stuff23:03
sonne|osxyou mean  non-linux23:03
sonne|osxno lets not optimize for weird platforms23:03
sonne|osxlets just use time23:04
@wikingsonne|osx: let's not23:04
@wiking/dev/urandom is totaly unix23:04
@wikingurandom is available in any normal unix system23:04
sonne|osxwiking: any unix has gettimeofday so /dev/urandom doesn't help at all23:04
@wikingthat is not from 1980s23:05
@wikingsonne|osx: yes it doesn23:05
@wikinggettimeofday and like23:05
@wikingis a very bad way to seed a prng23:05
@wikingas there's a high probabilyt23:05
@wikingthat it makes totaly not rng23:05
sonne|osxsure but it is just a seed23:05
@wikingso the whole purpose of PRNG is just being23:05
@wikingsonne|osx: yes that's why it's Prng23:06
@wikingand not RNG23:06
@wikingw/o a good seed23:06
@wikingno good random is possible23:06
@wikingwith any prng23:06
sonne|osxotherwise you would use /dev/random all the time23:06
@wikingsonne|osx: no23:06
sonne|osxand wait 20 seconds to get another integer23:06
@wikingsonne|osx: because u want Prng23:06
@wikingmeaning u wanna be able to have 'states'23:06
-!- sonne|osx [~sonne@f053047007.adsl.alicedsl.de] has quit [Quit: sonne|osx]23:08
shogun-buildbot_build #144 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/144  blamelist: Viktor Gal <viktor.gal@maeth.com>23:10
-!- hushell [~hushell@c-50-188-141-210.hsd1.or.comcast.net] has quit [Ping timeout: 272 seconds]23:24
-!- travis-ci [~travis-ci@ec2-23-20-94-251.compute-1.amazonaws.com] has joined #shogun23:34
travis-ci[travis-ci] it's Fernando Iglesias's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1431234823:34
-!- travis-ci [~travis-ci@ec2-23-20-94-251.compute-1.amazonaws.com] has left #shogun []23:34
shogun-buildbot_build #1967 of bsd1 - libshogun is complete: Failure [failed compile test]  Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/1967  blamelist: Viktor Gal <viktor.gal@maeth.com>23:45
-!- Saurabh7 [~Saurabh7@115.248.130.148] has quit [Ping timeout: 264 seconds]23:47
shogun-buildbot_build #128 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/128  blamelist: Viktor Gal <viktor.gal@maeth.com>23:49
shogun-buildbot_build #344 of FC19 - libshogun is complete: Success [build successful]  Build details are at http://buildbot.shogun-toolbox.org/builders/FC19%20-%20libshogun/builds/34423:54
-!- hushell [~hushell@8-12.ptpg.oregonstate.edu] has joined #shogun23:58
--- Log closed Fri Nov 22 00:00:35 2013