Open in new window / Try shogun cloud
--- Log opened Mon Jan 27 00:00:12 2014
-!- Krishna_ [80689984@gateway/web/freenode/ip.] has joined #shogun00:34
Krishna_I recently installed shogun 3.0.000:36
Krishna_The install process went smoothly00:36
Krishna_but I can't locate the .so or the .mex files00:37
Krishna_for the interface install00:37
-!- Krishna_ [80689984@gateway/web/freenode/ip.] has quit [Ping timeout: 245 seconds]04:28
-!- lisitsyn [] has quit [Quit: Leaving.]05:48
-!- tejasnikumbh [67157f4e@gateway/web/freenode/ip.] has joined #shogun06:30
tejasnikumbhHey guys06:30
tejasnikumbhAre any of the gsoc metors here06:30
tejasnikumbhI kinda have some ideas related to gsoc for this year which could be incorporated in shogun06:30
tejasnikumbhI'm posting them on the mailing list.06:31
tejasnikumbhin the replies06:31
tejasnikumbhPlease let me know about your availablity on irc06:31
-!- tejasnikumbh [67157f4e@gateway/web/freenode/ip.] has quit [Quit: Page closed]06:37
-!- tejasnikumbh [67157c4e@gateway/web/freenode/ip.] has joined #shogun08:52
-!- tejasnikumbh [67157c4e@gateway/web/freenode/ip.] has quit [Ping timeout: 245 seconds]09:02
-!- new_lido [~walid@] has quit [Quit: Leaving]10:44
-!- benibadman [~benibadma@] has joined #shogun11:48
-!- tejasnikumbh [67157c4e@gateway/web/freenode/ip.] has joined #shogun12:01
-!- tejasnikumbh [67157c4e@gateway/web/freenode/ip.] has quit [Quit: Page closed]12:06
-!- parijat [671b082a@gateway/web/freenode/ip.] has joined #shogun12:12
-!- parijat [671b082a@gateway/web/freenode/ip.] has quit [Client Quit]12:16
-!- lisitsyn [] has joined #shogun12:58
-!- lisitsyn1 [] has quit [Ping timeout: 260 seconds]12:59
-!- lisitsyn [] has quit [Read error: Operation timed out]13:04
-!- lisitsyn [] has joined #shogun13:05
-!- tom__ [2eda6d58@gateway/web/freenode/ip.] has joined #shogun13:27
tom__Hi all13:27
tom__A question about: HistogramIntersection Kernel13:28
tom__What type of features can I use with that ?13:28
tom__HashDocDot feats are derived from dot feats, does it match ?13:28
tom__HistogramIntersection should use dense features, is there any way to convert string feats, (or hashdocdot feats) into dense feats ?13:29
@wikingtom__: mmm well maybe it'd be better to convert HistogramIntersectionKernel to support other features13:37
tom__When StringFeatures refers to occurence histograms (of ngram etc ..), is it possbile to convert to dense features ?13:43
@wikinghaven't been working with stringFeats ...13:44
@wikingmaybe somebody else would have an opinion on this...?13:45
-!- tom__ [2eda6d58@gateway/web/freenode/ip.] has quit [Ping timeout: 245 seconds]14:14
-!- lisitsyn [] has quit [Quit: Leaving.]16:49
-!- parijat [671b082a@gateway/web/freenode/ip.] has joined #shogun17:37
-!- benibadman [~benibadma@] has quit [Ping timeout: 272 seconds]17:44
-!- parijat [671b082a@gateway/web/freenode/ip.] has left #shogun []18:01
-!- lisitsyn [~lisitsyn@] has joined #shogun18:18
-!- [paladin] [73f91219@gateway/web/freenode/ip.] has joined #shogun18:27
[paladin]Hey. I'm interested in discussing about participating in GSoC 2014. Who do I get in touch with ?18:29
lisitsyn[paladin]: hey18:46
lisitsynI can talk about it I think :)18:47
lisitsynis there anything you want to ask?18:47
[paladin]One. I haven't seen any official ideas page for GSoC 2014. So I was wondering what you guys had planned ?18:49
lisitsyn[paladin]: we are working on possible ideas these days18:50
[paladin]Two: do you guys have benchmarking methods implemented ?18:50
lisitsyn[paladin]: examples of ideas are GPs, random forests, MCMC and some frameworks18:51
lisitsyn[paladin]: no not really18:51
lisitsynwe had one in dotfeatures iirc18:52
[paladin]What about an automatic benchmarking tool ?18:52
lisitsynbut not as a general practice18:52
lisitsynif you want to propose such an idea it would be welcome if we are accepted :)18:52
lisitsynhowever we need to think what is it and how should it be done18:53
-!- HeikoS [] has joined #shogun18:53
-!- mode/#shogun [+o HeikoS] by ChanServ18:53
[paladin]Okay so should I approach when you have a page out to officially discuss ? Later ?18:56
lisitsyn[paladin]: well some kind of18:57
lisitsynalthough it would be cool if you propose your idea in the mailing list18:57
lisitsynso we can comment etc18:58
[paladin]okay. I think that would give me more time to sort of formalise it18:58
lisitsyn[paladin]: sure!19:00
[paladin]great. thanks.19:00
-!- [paladin] [73f91219@gateway/web/freenode/ip.] has quit []19:01
-!- [paladin] [73f91219@gateway/web/freenode/ip.] has joined #shogun19:02
-!- [paladin] [73f91219@gateway/web/freenode/ip.] has left #shogun []19:03
-!- killer_ [671b082a@gateway/web/freenode/ip.] has joined #shogun20:01
-!- iglesiasg [] has joined #shogun20:02
-!- mode/#shogun [+o iglesiasg] by ChanServ20:02
-!- zxtx [] has quit [Ping timeout: 248 seconds]20:15
-!- killer_ [671b082a@gateway/web/freenode/ip.] has quit [Quit: Page closed]20:15
-!- ChanServ changed the topic of #shogun to: Shogun Machine Learning Toolbox | Next Stammtisch Monday 2014-02-03 19-24hrs UTC |Devs are usually around between 10am-10pm UTC | When asking questions please be patient or use the mailing list. We always answer but it might take a while. | Channel logs
@sonney2kHeikoS, any news?20:27
-!- shogun-notifier- [] has joined #shogun20:32
shogun-notifier-shogun: Parijat Mazumdar :develop * 4390bbf / / (9 files):
shogun-notifier-shogun: minibatch kmeans added20:32
shogun-notifier-shogun: Parijat Mazumdar :develop * b7fe2e6 / src/shogun/clustering/lKMeans.cpp:
shogun-notifier-shogun: reverted to random algo20:32
shogun-notifier-shogun: Fernando Iglesias :develop * 6930eed / / (9 files):
shogun-notifier-shogun: Merge pull request #1827 from mazumdarparijat/fastkmeans20:33
shogun-notifier-shogun: mini-batch KMeans Added20:33
-!- thoralf [] has joined #shogun20:36
@iglesiasghello hello20:36
@sonney2khey thoralf20:37
thoralfDid I miss something? :)20:37
@sonney2kthoralf, heh20:38
lisitsynare we in that stamstststsich state?20:39
@sonney2klisitsyn, I am20:40
@sonney2klisitsyn, but I only have 20 minutes before being kicked out20:40
@sonney2kinternet here sucks big time20:40
@HeikoSsonney2k: hi!20:43
@HeikoSlisitsyn:  hi!20:43
@HeikoSsonney2k: I have two projects, talking to a guy about a third one20:44
@HeikoSI could do another one, but thats out of my time constraints ;)20:44
@sonney2kHeikoS, so it seems we have enough projects ...20:45
@HeikoSsonney2k: thats good!20:45
@HeikoSI will add two descriptions soon end of weekish20:45
lisitsynHeikoS: hey20:45
-!- benibadman [] has joined #shogun20:46
-!- benibadman [] has quit [Read error: Connection reset by peer]20:47
-!- benibadman [] has joined #shogun20:47
@sonney2kHeikoS, I cannot start doing stuff before next week20:48
@sonney2kHeikoS, internet sucks here so I am basically offline20:48
lisitsynsonney2k: where you at? ;)20:48
@sonney2kHeikoS, btw regarding funding - we had a paid developer at some stage. So I know that money is not the key to a OSS project's success20:49
@sonney2klisitsyn, Oberwiesenthal20:49
lisitsynsonney2k: is it dirk who is a paid developer?20:50
@HeikoSsonney2k: ok no worries there is still time20:50
lisitsynahh sonney2k having ski time?20:50
lisitsynI am zero-experienced but I tend to agree it is not about money, HeikoS sonney2k20:51
@HeikoSsonney2k: funding would still be nice, but yeah, money is not the thing20:51
@HeikoSbut continuity :)20:51
@sonney2klisitsyn, indeed but he wasn't the only one20:51
@HeikoSso if we could pay sergey, that would be cool20:51
@sonney2kHeikoS, money != continuity20:52
@HeikoSsonney2k: well up to some point it is20:52
@HeikoSjust hour wise20:52
lisitsynHeikoS: even in this case I could fail :D20:52
@sonney2kyes for a few months20:52
shogun-buildbotbuild #449 of FC19 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Parijat Mazumdar <>20:52
@sonney2kbut then it is just work20:52
@sonney2kand you loose interest in doing it the way you did...20:52
@sonney2kbut of course there are exceptions20:52
@HeikoSsonney2k: its worth a try20:53
@HeikoSbut yeah, as you said, gsoc first20:53
@sonney2klisitsyn, you are mr. chaos(tm) anywas :)20:53
lisitsynsonney2k: sure20:53
@sonney2kHeikoS, yes indeed20:53
lisitsynsonney2k: isn't it the way everything should work?20:53
@sonney2kGSoC gave use much more continuity :)20:53
lisitsynI have to admit I don't get continuity word here20:53
@sonney2klisitsyn, that is how our team works :D20:54
@sonney2kall entropy maximizers!20:54
lisitsynyeah sure I thought max ent is the way20:54
@HeikoSsonney2k: still would be good to have some less new-algorithm based person working on stuff20:54
@sonney2kyeah we had that - it is boring and at some point you wish you would do sth else20:55
@sonney2kbut of course this helped20:55
@HeikoSyeah one needs a good mix20:55
@sonney2kbut not regarding continuity20:55
@HeikoSand change targets every now and then20:55
@HeikoSI think we can only win by this :)20:55
@sonney2kML people don't like non-new-algorithm stuff20:55
@HeikoSwiking does :)20:55
@HeikoSsergey too20:55
@sonney2kand hardcore coders get bored with lowlevel stuff20:55
lisitsynHeikoS: we are not ml people ;)20:56
@HeikoSI dont see this so pessimistic and would see its good to try things,20:56
@sonney2kHeikoS, one pressing thing is getting the money btw20:56
@HeikoSevery now and then20:56
@sonney2kHeikoS, sure do it20:56
@HeikoSsonney2k: yeah just reading your email20:56
@sonney2kHeikoS, you just should not be 100% away20:56
@HeikoSsonney2k: I need that money too, fincance is pressing here20:56
@HeikoSsonney2k: I was 100% away from everything over the last months20:56
@HeikoSbut that wont happen anymore in the near/middle future20:57
@HeikoS(I hope at least)20:57
@sonney2kHeikoS, I know20:57
@HeikoSkind of crazy period here20:57
@sonney2kjust after the summit20:57
lisitsynyou must feel good now20:57
@sonney2kpretty bad :(20:57
@HeikoSdid not do anything else than working and sleeping20:57
@HeikoSI do feel free :)20:57
@sonney2kbtw guys we have this buildbot here showing very critical valgrind errors20:57
lisitsynHeikoS: how did you manage to concentrate? :D20:57
@HeikoSbut now there is all the postponed stuff waiting  ;)20:57
@HeikoSlisitsyn: well focus on one thing rather than many20:57
@sonney2kit is unfortunately green all the time20:57
lisitsynHeikoS: I constantly fail in this20:57
@HeikoSsonney2k: whats the problem with that?20:58
-!- HeikoS [] has left #shogun []20:58
-!- HeikoS [] has joined #shogun20:58
-!- mode/#shogun [+o HeikoS] by ChanServ20:58
@sonney2kHeikoS, it does valgrind on the tests20:58
@sonney2kHeikoS, and many have uninted memory reads / errors20:58
-!- HeikoS [] has left #shogun []20:58
-!- HeikoS [] has joined #shogun20:58
-!- mode/#shogun [+o HeikoS] by ChanServ20:58
@sonney2kHeikoS, and some of them fail20:58
@HeikoSsonney2k: I see20:59
@sonney2kon osx20:59
@HeikoSbad bad20:59
lisitsynoh I think I have a new idea to propose20:59
@sonney2kthey will kick me out of this lounge here in 1 min...20:59
lisitsyndictionary learning20:59
lisitsynthe problme is I am not an expert in that at all20:59
lisitsynis it a problem for any of you?20:59
@HeikoSsonney2k: whats the most important stuff to discuss for now20:59
shogun-buildbotbuild #450 of FC19 - libshogun is complete: Success [build successful]  Build details are at
@sonney2kHeikoS, well get the money from previous gsoc21:00
@sonney2kget into this years gsoc21:00
@sonney2kso proposal needs to be written21:00
@HeikoSsonney2k: ok, so we dont need more projects? just proposals?21:00
@sonney2kand ideas list in shape21:00
@HeikoSif I add two, we have some things?21:00
@HeikoSnew mentors?21:00
@HeikoSneural net? variational learning?21:00
@sonney2kI asked many people and some of them said yes so we should be fine21:00
@sonney2kwe have to finish it though but one thing at a time21:01
@sonney2kthen I would say21:01
@sonney2kget builbots to green21:01
@sonney2kand criticals fixed21:01
@sonney2kand then release...21:01
@HeikoSsonney2k: did you submit an amount along with the pdf invoices you sent?21:01
@sonney2kand then gsoc students Q/A as usual21:01
@sonney2kohh btw21:01
@sonney2kHeikoS, I thought I CC'ed you?!21:02
@sonney2kbut he actually did21:02
@sonney2kohh an btw workshop - > before eurypython in berlin21:02
@HeikoSsonney2k: phew load sof stuff21:02
@wikingstamtisch :)21:02
@HeikoSIll try to do something little every day ;)21:02
@wikingor wtf :> woah i've managed :>21:02
@sonney2kHeikoS, sure21:03
@HeikoSsonney2k: no you did not, do you still find the email?21:03
@sonney2kHeikoS, the guy attached the bill in his email21:03
@sonney2kHeikoS, so you have it too21:03
@HeikoSsonney2k: I don tknow whether I have to put a number21:03
@HeikoSor just the scans and they already have the sum21:03
@sonney2kIIRC all he wanted is that we prepend some bill to google21:03
@HeikoSwiking: hi there! how are things?21:03
@wikingHeikoS: pfff21:03
@sonney2kHeikoS, they have everything21:03
@wikingHeikoS: but thanks for asking :)21:03
@sonney2kHeikoS, wiking is in his final phase of phd21:04
@sonney2kso closer to suicide as usual ;)21:04
lisitsynoh shit I should write my masters thesis21:04
lisitsyngood you recalled21:04
@sonney2k(every phd student knows what I am talking about)21:04
@HeikoSwiking: haha ;) I share your pain21:04
@wikingHeikoS: thnx21:04
@sonney2kHeikoS, then I think I should try to get shogun properly into debian21:04
@sonney2kwe have sth in now21:04
@sonney2kbut no python etc21:04
@HeikoSsonney2k: I agree21:05
@sonney2kHeikoS, besser82 already gave us nice fedora packages21:06
@sonney2kso we are in much better shape than the years before21:06
@sonney2kHeikoS, would be nice if wiking could take care of osx stuff like brew / macports21:06
@sonney2kand we need our binary packages on the website somewhere21:06
@sonney2kI am being kicked out21:07
@sonney2ksee you guys21:07
lisitsynokay now people would notice my messages again :D21:07
thoralflisitsyn ;)21:07
@wikinglisitsyn: go and work on msc :)21:07
@wikingso what's with the licensing on the end21:07
lisitsynwiking: I don't know what is it about21:07
@wikingdo we have a conclusion, or the conclusion is that we have no conclusion :)21:08
@wikingokok i see21:08
lisitsynokay guys any suggestions for masters thesis? :D21:08
@HeikoSsonney2k: see you!21:08
lisitsynI have 3 months essentially :D21:08
@wikingwell then we should really think that if we get accepted for gsoc this year21:08
@HeikoSman deep mind just got bought by google21:08
@wikingthat maybe the students should send in their stuff with different license21:08
@wikingHeikoS: yes21:08
@HeikoSanother scary bit in their puzzle21:08
lisitsynwiking: I vote for BSD/MIT21:08
lisitsynfor new code21:09
@wikingno it's scary how much they paid for it21:09
@HeikoSwiking: deep mind is next door here and most people from the institute go there after phd21:09
@wikinglisitsyn: could go with it21:09
@wikingHeikoS: hehe thought so21:09
@HeikoSpretty cool caompany, would have wanted to go there21:09
@HeikoSbut now....aaargh21:09
@HeikoSwiking: whats with this bsd change?21:09
@HeikoSwiking: how much work is that?21:09
-!- zxtx [] has joined #shogun21:09
besser82HeikoS:  a f*cking bunch21:09
@wikingHeikoS: well it's more waiting than21:09
@HeikoSif we could do this kind of on the fly every now and then continously that would be great21:09
@wikingwork actually21:09
@HeikoSwiking: yeah thats what I mean21:09
@HeikoSand we can ask all code from now to be bsd21:10
@HeikoSor mit21:10
@HeikoSor whatever21:10
@wikingthat's not an issue21:10
besser82HeikoS, lisitsyn, wiking, adrin, iglesiasg, thoralf: Hey Folks!!!21:10
@wikingthe past code and their writers has to be contacted21:10
@HeikoSbesser82: hi! :)21:10
thoralfHey Bj?rn21:10
lisitsynis there anyone else (but HeikoS) who thinks we should kinda split up to a few smaller libraries?21:10
@HeikoSwiking: how much automatic would that be21:10
@HeikoSlisitsyn: is that a gsoc project?21:11
@wikingHeikoS: well i've done some basic stats by git21:11
@wikingit's doable21:11
lisitsynthis should solve licensing problem a bit21:11
lisitsynI mean we just replace parts21:11
@HeikoSwiking: ok, so maybe I will write an email and put it to disciussion this week21:11
lisitsyndo we need massive -> bsd conversion?21:11
@wikingi mean the core developers can right away assign to the right license21:11
@wikingthe problem is with contributors21:11
@wikingwho are gone21:11
@HeikoSlisitsyn: yeah I would just try things first and then see how it goes21:11
@HeikoSwiking: yep21:11
@wikingbut in worst case21:11
@HeikoSwould be good to have a map of things21:12
@wikingwe can do a rewrite of that code21:12
@wikingor let's say this is a good way to throw out code21:12
@wikingHeikoS: mindmap? :)21:12
lisitsynwiking: what's with berlin universitaet copyrights?21:12
@HeikoSwiking: yeah maybe later, I would first change everything thats possible and start changing new stuff21:12
@wikinglisitsyn: mmm has to be followed up21:12
@HeikoSand then discuss what to do with the rest21:12
@HeikoSbut guys, what about gsoc, whats your status on that? project ideas? ideas page, proposals? application, who manages?21:13
lisitsynwiking: is it enough to have author will to change anything?21:13
@wikinglisitsyn: yes21:13
lisitsynu sure?21:13
@wikinglisitsyn: but of course if there's an institution connected21:13
@wikinglisitsyn: then u have to get the institution to sign of the change as well21:13
lisitsynthen gsoc 2011 work is kind of lost21:13
@wikingHeikoS: i've promised to write the proposal21:13
@wikinglisitsyn: y?21:13
@HeikoSwiking: on what?21:13
besser82lisitsyn:  several smaller libs / plugin-stuff 4 sure!21:13
@wikingHeikoS: for gsoc... and put in shape the whole project ideas21:14
lisitsynbesser82: good to have people on your side :D21:14
@wikingHeikoS: will do this till wednesday21:14
besser82HeikoS:  me is mentoring SVM^bright && some cmake-based stuff21:14
@wikinglisitsyn: what's the problem with 2011 contribs?21:14
@HeikoSbesser82: cool!21:14
lisitsynwiking: I mean they are all signed as berlin stuff21:14
lisitsynas soeren was in TU at the moment21:14
@wikinglisitsyn: well we can get somebody in berlin to sign it off21:14
@wikinglisitsyn: yeah but there's alex and some other guys still there21:15
lisitsynah okay if it works I don't mind21:15
-!- benibadman [] has quit [Ping timeout: 245 seconds]21:15
@wikingso i dont really think that we cannot get somebody in TU to sign of a piece of paper21:15
lisitsynI have absolutely no clue how this works21:15
@wikinglisitsyn: i've checked up21:15
lisitsynthen we are ok21:15
@wikingwe need basically a statement21:16
@wikingfrom the parties that they are ok with the licensing change21:16
@wikingthat's all21:16
@HeikoSlisitsyn, wiking we can just ask and see then21:16
@wikingit's not rocket science21:16
@wikingso fucking let's go with BSD 2 liners21:16
@HeikoSnaywhayare: are you interested in having another project on this inter project comparison stuff?21:16
@HeikoSwiking: yeah I agree on that21:16
@wikingi'll scetch up the statement for that21:17
@wikingthat we can send around to license holders in shogun21:17
@HeikoSwiking: cool, maybe use the vlc stuff as inspiration21:17
@wikingand then we are good21:17
@wikingHeikoS: indeed21:17
@wikingHeikoS: get funding :)21:17
@HeikoSwiking: important part would be to maintain answers decisions and what parts are missing21:17
@wikingif docker got 15M usd...21:17
@wikingfuck man that pissed me off :)21:17
@HeikoSwiking: I know,21:17
shogun-buildbotbuild #45 of deb4 - python3 is complete: Failure [failed test python modular]  Build details are at  blamelist: Parijat Mazumdar <>21:17
@HeikoSwell we can do that, just need to work harder ;)(21:17
@wikingyeah as usual21:18
@HeikoSwiking: we are growing, so patiance and discipline will solve this ;)21:18
@HeikoSlast year, nobody was even talking about funding21:18
@HeikoSand the year before we did not even have unit tests21:18
@wikingHeikoS: heheh yeah sure21:18
@HeikoSnotebooks are also new21:18
@HeikoSso its going well!21:18
lisitsynHeikoS: DAS DISCIPLINE21:18
@wikinglisitsyn: :D21:18
-!- benibadman [] has joined #shogun21:19
@wikingbtw i have to load of some cash to some fundations21:19
@wikingso plz do something with shogun asap21:19
@wikingand then i'll transfer some money21:19
@HeikoSwiking: oh21:19
@HeikoSyeah everything formal should be ready21:19
@wikingHeikoS: not too much money21:19
@HeikoSnow we need to register21:19
@wikingbut still21:19
@wikingshoudl be enough as a started21:19
lisitsynwiking: you got a money you can't spend? ;)21:20
@wikinglisitsyn: nono21:20
@wikinglisitsyn: it's money that i have to spend for associations/fundation, i.e. nonprofit21:20
@wikingotherwise it's lost money21:20
@HeikoSwiking: wherefrom?21:20
lisitsynheh yeah interesting ;)21:20
@wikingbecause then the state will eat that money21:20
@wikingso i rather give that money for something i know...21:20
@wikingHeikoS: have a small company that has to do taxes21:21
@HeikoSwiking: I see21:21
@HeikoSvery cool!21:21
@wikingHeikoS: and i can assign freely 1% of my tax to nonprofit shiatz21:21
@HeikoShopefully next month we are set21:21
@wikingas said if i dont assign it to somebody21:21
@wikingthen it's just lost in the big state budget21:21
@wikinglet's hope we are still in time21:22
@wikingi've talked with vojtech21:22
@wikingabout some qp project21:22
@wikingit'll be a huge project21:22
@wikingi dont know if anybody will be up for that task21:22
@HeikoSthat would be cool21:22
@wikingbtw i could do gsoc this year (apparently i still have student status :DDD21:22
@HeikoShaha :)21:23
@HeikoSyou should rather mentor two students ;)21:23
@wikingok so lisitsyn r u going to lead the modularization ? :)21:24
@wikingi mean if we get into gsoc21:24
lisitsynwiking: yeah I think it would work for me21:24
@wikingthat should have a prio among the slots21:24
@wikingif there's a good student up for that task21:25
lisitsynwiking: I realized I am rather a gay^W^W a software developer than machine learning guy21:25
@wikinglisitsyn: :D21:25
@wikingbtw the whole story with dptrs would be still required even if we do modularization?21:25
lisitsynwiking: I think d-ptr stuff is useful anyways21:26
besser82wiking:  dtpr would be good in some parts, yes  ;)21:26
besser82lisitsyn:  :D21:26
@wikingi guess we would still benefit a lot from it21:26
lisitsynI believe once we get better compile time21:26
@wikingfor the swig interfaces21:26
lisitsynwiking: don't know21:26
lisitsynwe will see21:26
besser82wiking:  yes, swig would be a cleaner solution  then21:27
@wikinglisitsyn: yeah i think the generated cxx would be heaps smaller21:27
@wikingHeikoS: wha't swith that view thing atm?21:27
besser82wiking: and generation of them cxx wouldn't take 1.5GBytes each anymore  :-P21:27
-!- dsockwell [~twgs@] has quit [Ping timeout: 272 seconds]21:28
lisitsynoh guys guys21:28
@HeikoSwiking: the guy did not finish it unfortunately21:28
lisitsynI have an idea21:28
@wikingHeikoS: what's missing?21:28
@HeikoSwiking: the solution was nice though21:28
@HeikoSshould not be hard to finish it21:28
lisitsynlets put whole shogun code to one .h21:28
@wikingHeikoS: if u tell me what's missing i could give it a go21:28
@HeikoSwiking: some playing around with it to see it works and fix copy constructors of some feature classes21:28
@wikinglisitsyn: !!!21:28
lisitsynwhat's your bet on required RAM?21:28
@wikingHeikoS: indeed21:28
@HeikoSwiking: have a look at the current PR of the guy21:28
@wikingHeikoS: will do21:28
@HeikoSwiking: and then change the x-validation class to use it as a test case (and write unit tests :)21:28
@HeikoSand then it should be ready to be used by others21:28
lisitsynI think 30gb should be the number :D21:28
@HeikoSand if you dont change the copy-by-reference objects, it is threadsafe21:29
@wikingHeikoS: ok so not muc :)21:29
@wikingHeikoS: well i might just skip xvalidation21:29
@wikingHeikoS: but test it in other places :PPP21:29
@wikingHeikoS: openmp created shiatz21:29
-!- dsockwell [~twgs@] has joined #shogun21:30
@HeikoSwiking: I think it would be good to test it with xval since its a good test case and there are already things relying on it so we can see whather stuff breaks21:30
@HeikoSwiking, lisitsyn I have to go now. See you tomorrow!21:31
@wikingHeikoS: cyaz21:31
-!- HeikoS [] has quit [Quit: Leaving.]21:31
lisitsynwiking: I want to have dictionary learning in shogun too21:34
lisitsynmay be I could mentor it somehow21:34
lisitsynat least some ksvd21:34
@wikinglisitsyn: we can steal the code :)21:34
lisitsynyeah sure21:34
lisitsyneverything is already implemented21:34
@wiking'steal' = opensource21:34
@wikingyeah so that shouldn't be a biggie21:35
@wikingi mean we can put together such a slot as well21:35
@wikingwhere u dont need too much of a haxor skill21:35
@wikingrather just do fast integration of some crucial basic algo21:35
lisitsyncause plugins is a hardcore thing21:35
@wikingthat are atm missing from shogun21:35
@wikingyeah that's like pretty fucking crayz21:35
lisitsynwe ain't gonna have a student for that21:36
@wikingi can come as a student as i told :DDD21:36
@wikingbut imo i wont have time21:36
@wikingthoralf: i see it's rather a longer fix21:37
@wikingthoralf: there were a lot of code redundancy in the lib* implementation21:37
@wikingand as it's a matlab -> c port21:38
@wikingi wouldn't be too surprised of such 'little' bug21:38
thoralfwiking: I hope I finally convinced Michal ;)21:38
-!- travis-ci [] has joined #shogun21:39
travis-ci[travis-ci] it's Fernando Iglesias's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []21:39
thoralfwiking: Let me look to the other classes if they have similar issues...21:39
@wikingthoralf: lib*bmrm are quite having the same codebase21:40
@wikingthey only have minor differences21:40
thoralfHmm, handling of beta is quite different in libp3bm and libncbm.21:46
thoralfBut libppbm.cpp seems to suffer the same issue.21:47
thoralfline 26521:47
naywhayareHeikoS: sure; we have the framework in place and it could definitely be improved21:49
naywhayareoh, he's not even here.  I should check first ...21:49
thoralfwiking: Thanks for pointing that out.21:51
@wikingnaywhayare: which/what fw?21:54
@wikingthoralf: nw... in the past i've hacked a bit that code21:55
@wikingto make it a bit more shogun21:55
naywhayarewiking: last year as a GSoC project mlpack had a student who made an automatic benchmarking system22:02
naywhayareI met HeikoS at the mentor summit and we talked about how it would be useful for both of our libraries -- or really all ML libraries in general22:02
naywhayarehere's the last run of benchmarks we had...
naywhayareanyway, it needs better documentation, better details, probably a better interface22:03
naywhayareI think HeikoS was interested in adapting the system so it could be used by shogun22:03
@wikingoh cool22:03
@wikingnaywhayare: is it hardwired with mlpack ?22:03
naywhayareI also need to rebuild the benchmarks for your latest release but a paper deadline is on the 31st so it won't be done until after that...22:03
naywhayareno, it's library-agnostic22:03
naywhayarewhat the framework really is is just a big python script that runs things and compares how long they take22:04
naywhayareit has a configuration file that defines what to run, what datasets to use, and how to run it22:04
naywhayareand then it has little python scripts individually written for each library's implementation of each method22:04
-!- iglesiasg [] has quit [Quit: Leaving]22:05
@wikingnaywhayare: and python runs like a binary or if there's a python interface u use that?22:05
@wikingnaywhayare: cool it seems there'a lot of stuff we need to fix :P22:07
@wikingin shogun22:07
@wikingi'm just checking GMM results22:07
@wikingnaywhayare: do u have somewhere the code released?22:08
naywhayarewiking: yeah, here's the repo link...22:08
naywhayareit wasn't really part of mlpack so we put it in our configuration directory for our build server22:08
naywhayareand we have it all integrated with the build server so that it's easy to run tests22:08
naywhayarehere's some further documentation22:09
@wikingok cool22:09
@wikingi'll try to run this myself22:09
naywhayarelike I said it's not completely polished22:10
naywhayarewe did our best to write the scripts for other libraries to use them as well as possible22:10
@wikingno worries22:10
thoralfwiking: libppbm has the same issue - just proven it in a unit test.22:10
@wikingthoralf: \o/22:10
naywhayarethe cover tree is not used for shogun k-nearest-neighbors because john langford's code has a bug he has yet to fix and it returns incorrect results (I think I mentioned this in here a while back; he still hasn't fixed it)22:11
@wikingnaywhayare: maybe we could start a separate repo for this on github?22:11
naywhayarealso, when you see '-' instead of 'failure' for the results, it means the test wasn't run for that particular dataset22:11
@wikingnaywhayare: and then we can get travis to do all the testing? :P22:11
naywhayarethat's just a configuration file issue and I'm in the process of fixing them little by little...22:11
@wikingnaywhayare: how long did it take for u to run the benchmark22:12
naywhayareheh.  the mlpack one takes two weeks22:12
naywhayarethe other libraries tend to take 3-5 days22:12
@wikingoh woah :D22:12
naywhayarewith mlpack we do memory profiling too22:12
@wikingnaywhayare: with?22:12
@wikingnaywhayare: do u use some fancy malloc library or the base system?22:12
naywhayarejust valgrind, I'm not 100% sure on the details22:13
naywhayareif you click the rightmost button for a method, then you can open up the 'memory results'22:13
naywhayareand it shows you memory usage for mlpack during the run of the algorithm22:13
@wikingi see22:13
@wikingwould be great to have that for all the libraries of coursE :P22:13
naywhayareyeah; we did what we could, but time is so little that it's really hard to keep the effort going22:14
@wikingbut then again that's future22:14
naywhayareI have enough trouble just keeping up to date with the five or six libraries we do test22:14
@wikingnaywhayare: totally understand22:14
naywhayarebecause every time a new release happens I have to recompile it on my test system, then ensure the python scripts still work22:14
naywhayare...then wait 3-5 days for the results to be done :)22:14
@wikingnaywhayare: do u think we could get a separate repo for this22:14
@wikingso that anybody can contribute easily22:15
naywhayareyeah, I mean, it's open-source.  =)22:15
@wikingi'm sure ppl here in shogun would be keen to contribute into it22:15
naywhayareideally the student should own the github repo, since it was his work and he knows the most about it22:15
@wikingnaywhayare: yeah no i mean that we could say that we create an official repo at github for it22:15
@wikingand then jsut have collaborators for the same repo22:15
@wikingand then PRs can be sent etc22:15
@wikingnaywhayare: r u still in touch with the student?22:16
naywhayareI'm getting in touch with him about it (his irc nick is 'marcus_zoq' and he hangs out in #mlpack or you can pm him); I'll let you know what he says22:16
naywhayareyeah, we still talk regularly, which is good22:16
@wikingcould u please ask him if he would do this?22:16
shogun-buildbotbuild #46 of deb4 - python3 is complete: Failure [failed test python modular]  Build details are at  blamelist: Fernando Iglesias <>22:16
naywhayareI just did, don't worry =)22:16
@wikingcool great!22:16
naywhayarereally github is a better place anyway than hidden somewhere inside our jenkins configuration22:16
@wikingheheh yeah22:16
@wikingjust with a small link to that page in README.md22:17
@wikingand that's all22:17
@wikingafter that issues and PRs will fly anyways22:17
naywhayarewe submitted the project to the MLOSS workshop but the reviewers bashed it because they were more interested in comparing the accuracy of algorithms instead of the runtime22:17
naywhayarewhereas for the most part we try to ensure that we are benchmarking the exact same algorithms against each other22:17
@wikinghehehh yeah man22:17
@wikingi mean reviewers are like a joke sometimes22:17
@wikingi mean there's like 8 out of 10 papers22:18
naywhayarenah, it wasn't a bad review, his points were valid, but sort of missed the point of what we were doing a bit22:18
@wikingwhere they compare algos22:18
@wikingbut then u know that they fine tuned the last bit of their algo in the paper22:18
@wikingwhile the ones they used for comparison22:18
@wikingjust used as is (default parameters etc)22:18
naywhayareyeah, naturally.  it's really difficult to get a clear picture like that22:18
@wikingso really a thing like this would be great to have22:18
@wikingjsut would be great to automate a lot of part of it22:19
@wikingi mean the test generation22:19
naywhayareso you can use kernel PCA for denoising and alex smola's big paper on it from 1999 has some awesome results... but that paper doesn't mention that when you do image denoising with kpca you still end up with outliers that have incredibly poor quality22:19
@wikingalex alex22:19
@wikingi wonder where's he now22:19
naywhayarethat was a thing I learned this weekend.  overall, the average reconstructed image quality is great!  but there are just some terrible outliers sometimes.  I wonder if anyone's investigated that22:19
@wikingused to be in canberra22:19
naywhayareeither CMU or Google or Yahoo22:19
naywhayareI met him at ICML... his poster was next to mine22:20
naywhayareI thought that would be great for the publicity of my poster22:20
naywhayarebut actually everyone just talked to him and not me :(22:20
@wikingyeah usual22:20
@wikingsorry to hear that though22:20
@wikingbtw how to u store the results?22:20
@wikingi mean is it stored in a db or something?22:20
naywhayareyeah, sqlite db22:20
@wikingok cool22:20
@wikingso basically on the end u can generate historical data as well22:21
naywhayareyeah, although I don't think we have the support to do that well22:21
naywhayarenot right now at least22:21
@wikingyeah sure22:21
naywhayarebut it can be done; there's just nothing in there that will generate those graphs22:21
@wikingbut if it's stored with a timestamp22:21
@wikingit's really just up to a php/python script to do that22:21
naywhayare there's our build server.  if you open the 'z benchmark' category then you can see each job that we run22:21
naywhayareso when a new release happens, I update it on the system that will be running on the benchmarks (compile with -O3 and whatever), then click 'start job' on jenkins22:22
@wikinggreat stuff22:22
@wikingi think there's a lot of great stuff that can be done with this22:22
@wikingi mean on the end if u think about it... if this benchmark gets more wide22:23
@wikinglibrary releasers will themselves generate the output22:23
@wikingof their new release22:23
naywhayareyeah, I agree22:23
naywhayarebut again it gets really difficult to compare algorithms sometimes22:23
naywhayareI can compare nearest-neighbor pretty easily because it's exact, but... do I compare it with kd-trees, or with the library's default trees?22:24
@wikingwell of course22:24
naywhayareis the goal of the benchmark to show what a typical user might actually write in code, or to show the actual capabilities of the library?  it gets difficult to answer the question cleanly22:24
@wikingbut there's always going to be problems like that22:24
naywhayarebecause it may be that while I can use mlpack's AllkNN object, it's 5x as fast if I write a far more complex use of some other class22:24
@wikingbut we dont want to solve everything with this benchmark22:25
@wikingrather to show at least basic stuff22:25
@wikingand compare them22:25
naywhayareyeah.  I see those types of problems as a potential source of arguments that I really want to avoid.  I don't want to hurt anyone's feelings :)22:25
@wikingnono i mean i think everybody understands this22:25
naywhayarethere definitely needs to be a lot of clarification on what exactly each benchmark is doing, because it's unclear right now (for instance it just says "ALLKNN")22:25
@wikingbut this is good for everybody both users and library developers22:25
@wikingat least get the bigger picture22:25
naywhayareit's super useful for an internal mlpack development tool, though22:25
@wikingso it's really good ... only that we need to get some more people on board22:26
@wikingbut i think that will not be a problem22:26
naywhayareyeah.  I haven't had the time that I really need to put into this to make it as nice as I'd like it to be22:27
naywhayareanyway a few good GSoC students can get a lot done :)22:27
@wikingwell dont try to do all yourselves :)22:27
@wikingjust put together a repo for it22:27
@wikingand fire up the issues22:27
@wikingi think things will get started22:27
naywhayaremy hope is that Marcus will want to run that22:27
naywhayareafter all, it's his work :)22:28
@wikingheheh yeah22:28
@wikingnaywhayare: the good thing would be to generate a standard API where u can submit your results22:28
@wikingvia http22:28
naywhayareI haven't yet approached the scikit-learn guys.  I told myself I'd do that when I had time but it looks like I'll never actually have time ever again22:28
naywhayareyeah, but the benchmarks aren't useful if they're not run on the same system22:28
naywhayarewhich is why we limit our runs to one specific system22:29
@wikingbut one can run the whole test22:29
@wikingon another system22:29
naywhayareah, yeah22:29
@wikingthat's useful as well22:29
@wikingthen u see what's the difference between machine A and machine B22:29
@wikingthere's tons of things that could be done with this22:30
@wikingso let's get marcus on board :)22:30
naywhayareyeah.  making a good interface to be able to actually parse all that data is the really tricky part22:30
naywhayareor, I'm bad at interfaces :)22:30
@wikingi mean honestly it'll be a great thing if this gets picked up around 201522:31
@wikingthat's my optimistic opinion :P22:31
@wikingthere's still a lot to work on, but i believe everybody would be interested in this if it would get a little bit more attention22:31
naywhayaresonney2k pointed out that they tried a similar thing some years ago with but did not see widespread adoption22:32
@wikingone never knows when things go haywire :P22:33
-!- benibadman [] has quit [Remote host closed the connection]22:35
@wiking"US reaches deal with Internet companies on requests to reveal demands for customer information"22:35
-!- iglesiasg [] has joined #shogun22:35
-!- mode/#shogun [+o iglesiasg] by ChanServ22:35
@wiking2 more years and there's gonna b like usa intranet and rest of the world internet if it continues like this22:36
naywhayareI'd like to hope Edward's Snowden's revelations will cause some kind of useful reforms.  but I'm cynical and I doubt it...22:38
@wikingnaywhayare: yeah like with obama and change, right? :D22:38
@wikingin retrospect that was like one of the biggest scam of the states :)22:39
naywhayarein a country as big as ours (or really any big country) it's very easy to be cynical about politics because a single person's opinions can't really effect any change.  which is very frustrating22:40
naywhayareon a very local level, though, like a neighborhood / city level, I've had good interactions with local politicians22:40
@wikingwell things are everywhere the same...22:41
@wikingi bet there's no one single person here who could tell the opposite22:41
naywhayarehah, probably22:41
@wikingwhenever i meet somebody from another country22:42
@wikingthere's one single thing that everybody has in common22:42
@wikingthings are not going well in politics :)22:42
@wikingi remember meeting this guy from brasil tell me stories of his own politicans, and in the meanwhile i was like 'yeah i know the story, they did the same...'22:43
@wikingso it's all the same imo22:43
@wikingi bet iglesiasg could share some very positive stuff from the royal family of spain :PPPPP22:44
naywhayareI was in spain recently, I thought it was really nice :)22:44
naywhayarebut I didn't deal with the politics at all... :)22:44
@wikingyeah that's the rule i always follow when i'm abroad22:44
@wikingignore local politics22:44
@wikingthings are just heaps better then :)22:44
@wikingso lately i follow that rule for my own country's politics... so far so good :22:45
@wikingman i fucking hate hadoop sometimes :)22:46
@wikingthe moment when your 4th job out of 5 jobs dies...22:46
@wikingafter 12 hours :)22:46
@wikingof running time22:46
@wikingwith a big fat exception22:47
@wikingthat basically doesn't really say anything22:47
@wikingeven an OOM would be more meaningful22:47
-!- HeikoS [] has joined #shogun23:12
-!- mode/#shogun [+o HeikoS] by ChanServ23:12
-!- HeikoS [] has quit [Client Quit]23:14
thoralfwiking: Err.  I found a funny bug, that actually has no effect, but maybe you've got an idea how to fix it:23:23
thoralfevery lib*bm defines a BufSize23:23
thoralfWhen calling any solver, the "local" version is used.23:24
thoralfBut when calling libbmrm::clean_icp() and adding an assertion there, the libbmrm-version is used, no matter from which solver it's been called.23:24
@wikingthat's most probably a bug introduced by me23:31
@wikingbecause before that the code was just copy pasted among the 3 .cpp23:31
@wikingso i wanted to remove some redundancy23:31
@wikingbut then i guess i've made there a mistake23:32
thoralfActually is has no effect, but it's irritating as soon as you use BufSize from somewhere else. ;)23:32
@wikingsorry for that23:32
-!- shogun-notifier- [] has quit [Quit: transmission timeout]23:33
thoralfCan it be fixed by using the right qualifiers for BufSize?  Something like extern uint BufSize instead of static?23:34
@wikingmmm nasdaq is down 3 days in a row :<23:34
@wikingthoralf: yeah that should work23:34
thoralfI'll check.23:35
thoralfwiking: Err.  NASDAQ?  What happened?23:38
thoralfDidn't read news for days...23:39
@wikingthoralf: mmm well since friday some countries' market is under siege23:39
@wikingso it's all down now :)23:39
@wikingbut mainly it's FED's fault.... they are starting to pull the plug :)23:40
@wikingtapering started basically :)23:41
-!- iglesiasg [] has quit [Quit: Leaving]23:50
--- Log closed Tue Jan 28 00:00:13 2014